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INTRODUCTION 

This  project  addresses  the  need  for  research  on  service  delivery  approaches  for  Service 
members  with  combat-related  physical  or  psychiatric  symptoms,  including  Posttraumatic  Stress 
Disorder  (PTSD)  and/or  post-concussive  symptoms.  As  a  primary  care  encounter,  the  post¬ 
deployment  health  reassessment  (PDHRA)  process  is  critical  to  force  health  protection  efforts,  and 
the  improvement  of  this  process  has  been  the  focus  of  the  work  reported  here.  This  work  is 
informed  by  a  previous  evaluation  of  the  PDHRA  process,  a  collaborative  effort  between  Vanderbilt 
University  (VU)  and  Force  Health  Protection  and  Readiness  (FHP&R).  The  final  report  of  the 
evaluation  conducted  under  this  previous  contract  is  available  on  Defense  Technical  Information 
Center  (DTIC)  at  http://handle.dtic.miI/100.2/ADA528063. 

This  project  has  two  goals.  (1)  Develop  an  evidence-based  training  program  for  providers  who 
deliver  deployment  related  assessments.  (2)  Evaluate  the  feasibility  and  efficacy  of  a  targeted 
training  and  feedback  program  on  primary  care  provider's  interview  and  clinical  communication 
patterns  related  to  Service  member  behavioral  health  condition  identification  and  referrals.  Three 
research  aims  include:  (1)  Explore  providers'  perceptions  of  workshop  acceptability  and  feasibility; 
(2)  Assess  impact  of  workshop  on  Service  member  ratings  of  provider  communication  skills;  (3) 
Determine  efficacy  of  workshop  on  provider  identification  of  Service  members'  psychosocial 
concerns. 

To  accomplish  these  aims,  a  training  workshop  that  incorporates  experiential  learning  strategies 
and  evidence-supported  characteristics  of  high  quality  communication  training  programs  was 
developed  and  piloted  at  4  sites  to  a  total  of  23  providers  during  Year  3.  In  addition  to  the  providers 
who  attended  the  workshop,  data  were  collected  from  28  providers  who  will  serve  as  a  control 
group  (they  did  not  attend  the  workshop),  resulting  in  a  quasi-experimental  research  design  that 
controls  for  common  threats  to  validity  such  as  changes  that  occur  over  time  independent  of  the 
intervention.  Pre-  and  post-outcome  measurements  included  brief  post-PDHRA  surveys  completed 
by  the  Service  member  on  tablet  computers  (anonymously)  and  provider  immediately  after  each 
PDHRA  interview.  These  surveys  were  administered  at  all  sites  for  a  period  of  time  before  and  after 
the  workshop  (typically  2-3  days  before  and  2-3  days  after),  and  were  subsequently  linked  to  actual 
PDHRA  data  to  facilitate  evaluation  of  workshop  efficacy.  Additional  measures  included  a  program 
manager  interview  and  surveys  completed  by  providers  before  and  after  the  workshop.  Year  4 
activities  have  focused  on  analysis  of  the  data  collected  in  Year  3.  Results  of  these  analyses  show 
that  compared  to  control  group  providers,  providers  who  attended  the  workshop  received  higher 
ratings  of  communication  skills  from  Service  members;  identified  more  psychosocial  concerns 
among  Service  members  who  reported  such  concerns  on  VU's  anonymous  survey;  and  reported  an 
increased  number  of  concerns  on  PDHRAs  (while  also  decreasing  the  number  of  medical  referrals 
but  increasing  referrals  to  Military  One  Source). 

The  project  is  a  cooperative  effort  among  VU,  FHP&R,  and  Purdue  University  (VU's  subcontractor). 
The  project  period  of  performance  is  30-SEP-09  to  31-JUL-14.  This  report  summarizes  Year  4  (30- 
SEP-11  to  29-SEP-13)  progress  on  scope  of  work  (SOW)  activities,  key  research  accomplishments, 
and  reportable  outcomes.  We  conclude  by  summarizing  results  to  date  and  projecting  work  to  be 
accomplished  through  the  remainder  of  the  project. 
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BODY  OF  REPORT 
Vanderbilt  University  SOW  Tasks 

Task  1.  Timing  of  Approvals  and  Institutional  Review  Board  (IRB) 

Task  1  activities  are  oriented  to  ensuring  that  all  proper  approvals  and  IRB  activities  are  completed 
in  a  timely  manner,  so  that  the  provider  intervention  and  other  research  activities  proceeds 
according  to  schedule.  In  Year  4,  IRB  approvals  were  limited  to  continuing  review  submissions,  all  of 
which  were  approved  by  both  VU  and  MRMC  IRBs  by  12-JUN-13.  One  of  these  submissions  (the 
Aim  2  Secondary  Analysis  Protocol)  involved  the  reporting  of  a  non-compliance  with  protocol  event 
in  which  AFHSC  had  inadvertently  sent  Service  member  Social  Security  Numbers  to  VU.  On  12-JUN- 
13,  the  Army  MRMC  IRB  sent  documentation  of  its  acceptance  of  the  report  and  ruling  that  no 
additional  actions  pertaining  to  the  protocol  noncompliance  were  required. 

During  Year  4,  VU  also  requested  and  received  approval  to  add  a  third  no-cost  extension.  Approval 
for  a  three  month  extension  was  received  on  16-OCT-13.  On  29-OCT-13  our  portfolio  manager 
recommended  that  we  instead  be  granted  a  6  month  no-cost  extension  and  requested  that  VU 
submit  a  modified  SOW.  We  received  notice  that  the  no-cost  extension  modification  had  been 
executed  on  4-NOV-13.  The  SOW  modification  was  submitted  on  07-NOV-13  and  is  currently  under 
review.  (See  Appendix  A  for  the  executed  no-cost  extension  modification,  the  SOW  modification 
currently  under  review,  and  the  two  SOW  modifications  previously  executed  in  Year  3). 

Task  2  (Aim  1).  PDHRA  Focus  Groups 

The  original  goal  of  Task  2  was  to  conduct  focus  groups  of  key  stakeholders  involved  in  the  PDHRA 
process,  and  to  analyze  the  resultant  data  with  the  intention  of  identifying  key  elements  for  training 
interventions  relevant  to  content,  format,  and  implementation.  However,  due  to  the  impact  of  the 
National  Defense  Authorization  Act  (NDAA)  legislation  introduced  between  the  time  of  the  original 
proposal  and  the  start  of  work,  FHP&R  and  VU  agreed  not  to  conduct  the  focus  groups  pending 
Department  of  Defense  (DoD)  efforts  related  to  the  legislation.  During  Year  2,  VU  received  approval 
from  MRMC  to  eliminate  the  focus  groups  from  the  SOW  (See  Appendix  A  second  section). 

Task  3  (Aim  1).  Secondary  Analysis  of  PDHRA  Data  from  Previous  Evaluation 

The  stated  goal  of  Task  3  is  to  conduct  secondary  analyses  of  the  PDHRA  data  obtained  during  VUs 
previous  DoD-funded  evaluation  of  the  PDHRA  process,  with  a  focus  on  identifying  provider  factors 
that  contribute  to  candid  Service  member  reporting  of  behavioral  health  concerns  and  to  Service 
member  acceptance  of  associated  referrals.  The  resulting  information  was  to  be  utilized  in  the 
development  of  the  training  and  feedback  intervention.  While  these  tasks  were  originally  scheduled 
for  Year  1,  delays  in  receiving  the  data  caused  them  to  become  later  year  activities  (See  Annual 
Reports  for  Years  1,  2,  and  3  for  a  description  of  these  delays). 


All  data  sets  were  merged  and  cleaned  prior  to  the  beginning  of  Year  4.  During  Year  4,  analyses  of  these 
data  have  been  ongoing,  with  preparation  of  publications  derived  from  this  analysis  planned  in  Year  5. 
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Task  4  (Aim  2).  Training  and  Feedback  Intervention  Efficacy  Study 

The  activities  listed  under  Task  4  address  the  central  goal  of  VU's  research,  which  is  to  develop  and 
test  the  efficacy  of  a  targeted  training  and  feedback  intervention  designed  to  help  providers 
increase  Service  member  reports  of  behavioral  health  concerns  and  Service  member  acceptance  of 
referrals  for  further  assessment.  Substantial  changes  to  these  original  tasks  were  made  and  are 
described  in  the  approved  SOW  modifications  included  as  Appendix  A. 

The  Year  3  Report  describes  the  development  and  implementation  of  the  intervention  at  four  Army 
installations:  Fort  Campbell,  Fort  Stewart,  Fort  Carson,  and  Fort  Bliss.  In  Year  4,  Task  4  work  has 
focused  on  Task  4h,  data  management  and  analysis  to  explore  workshop  feasibility  and  efficacy. 

Data  Management  and  Analysis 

Intervention  feasibility  and  efficacy  were  measured  using  a  combination  of  data  collected  by  VU  at 
site  visits  and  existing  data  collected  by  the  military: 

Measures  Developed  and  Administered  by  VU* 

1.  Provider  Background  Survey.  This  initial  background  and  self-efficacy  survey  was  completed 
by  providers  before  the  training.  When  possible,  this  was  administered  to  providers  not 
participating  in  the  training  as  well. 

2.  Provider  Workshop  Evaluation.  This  survey  was  completed  by  providers  after  the  training. 

3.  Provider  Post-Workshop  Self-Efficacy  Survey.  This  survey  included  the  same  battery  of  self- 
efficacy  questions  as  was  included  in  the  Provider  Background  Survey  and  was  completed  by 
workshop  participants  2-3  days  after  the  training. 

4.  Provider  Post-PDHRA  Survey.  This  brief,  3-item  form  was  completed  by  providers 
immediately  after  each  PDFIRA  encounter. 

5.  Service  member  Survey.  A  brief,  voluntary  satisfaction  survey  was  completed  by  Service 
members  immediately  after  the  PDFIRA  interview. 

*See  the  Year  3  Report  for  copies  of  these  measures;  descriptions  of  the  use,  modification,  and 
validation  of  previously  published  scales  in  these  instruments;  and  summaries  of  the  data  collected. 

Existing  Data 

1.  PDHRA  Data.  Including  all  PDFIRAs  completed  during  the  study  time  period  (3  months 
before  and  3  months  after  the  intervention)  along  with  all  pre-existing  PDFIRAs  associated 
with  the  PDFIRAs  collected  in  the  timeframe. 

2.  PDHA  data.  All  pre-existing  PDFIAs  associated  with  the  PDFIRAs  collected  in  the  timeframe 
described  above: 

3.  Health  Care  Encounter  (HCE)  Data.  All  pre-existing  HCE  data  associated  with  the  PDHRAs 
collected  in  the  timeframe  above. 
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Final  datasets  of  PDHRA,  PDHA,  and  HCE  data  were  received  from  AFHSC  on  29-APR-13.  (See 
Quarterly  Reports  for  a  description  of  circumstances  that  delayed  VU's  receipt  of  this  data). 
Following  the  receipt  of  these  data  sets,  all  data  were  linked  to  create  a  master  file  to  be  used  for 
subsequent  analyses,  with  linkage  being  complete  by  ll-MAY-13. 

Table  4.1  summarizes  Year  4  analysis  activities  by  research  aim  and  methodology.  More  detailed 
information  on  data  management  and  data  sources  can  be  found  in  Appendix  B. 


Table  4.1  Description  of  Detailed  Research  Aims,  Data  Sources,  and  Analyses 


Research  Aim 

Data  Set(s)  Used 

Analysis  Conducted 

1.  Explore  providers' 
perceptions  of  workshop 
acceptability  and  feasibility 

Provider  Workshop  Evaluation 

Descriptive  analyses  and 
quotes  from  open-ended 
comments. 

2.  Assess  impact  of 
workshop  on  Service 
members  ratings  of  provider 
communication  skills 

Service  member  Survey, 
incorporating  the  use  of  several 
previously  published  validated 
scales*,  which  were 
subsequently  validated  for  use 
in  the  military  population  by 

VU. 

Analyses  were  conducted 
using  hierarchical  linear  model 
regressions  accounting  for 
clustering  by  provider  to 
estimate  the  differences  by 
provider  group  (participated 
and  did  not  participate  in 
workshop)  and  time  (pre-  and 
post-workshop). 

3.  Determine  efficacy  of 
workshop  on  provider 
identification  of  Service 
members'  psychosocial 

concerns 

(a)  Provider  Post-PDFIRA  Survey 
Service  member  Survey 

Creation  of  a  scale  to 
represent  provider 
identification  of  Service 

member  behavioral  health 
concerns  followed  by  Poisson 
loglinear  regression  to  analyze 
differences  by  provider  group 
(participated  and  did  not 
participate  in  workshop)  and 
time  (pre-  and  post¬ 
workshop). 

3.  (cont.) 

(b)  PDHRA 

Difference  in  Difference 

Analysis  to  compare  pre-  and 
post-  workshop  changes  for  the 
two  groups  of  providers 
(participated  and  did  not 
participate  in  workshop). 

*A  detailed  description  of  these  published  scales  along  with  a  summary  of  VU  modifications  is 
included  in  the  Year  3  Report. 


The  results  of  these  analyses  were  presented  at  the  American  Psychological  Association  Annual  Meeting 
on  31-JUL-13,  and  in  more  detail  at  the  FFIP&R  Psychological  Flealth  Forum  on  19-SEP-13.  Appendices  C 
and  D  contain  the  results  of  VU's  analyses  to  date  in  the  form  of  presentation  slides. 
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In  Year  5,  analysis  activities  are  expected  to  include: 

•  Exploration  of  the  impact  of  the  training  workshop  on  providers'  self-reported  confidence 
and  self-efficacy  relating  to  communication  during  PDHRA  interviews. 

•  Continued  exploration  of  workshop  impact,  building  analytic  models  that  incorporate  PDHA 
and  HCE  data. 

o  Identify  covariates 

o  Limit  analyses  of  PDHRAs  to  those  closest  to  the  date  of  the  workshop 
o  Determine  the  best  method  to  explore  impact  of  the  workshop  on  PDHRA  provider 
documentation  by  Service  member  self-reported  problems 

Task  5.  Expert  Panel  Meetings 

The  purpose  of  the  Expert  Panel  meetings  is  to  ensure  that  intervention  development  is  fully 
informed  by  the  needs  and  resources  of  all  Service  Branches  and  Components.  Contributions  from  the 
Expert  Panel  helped  shape  training  development  during  Years  2  and  3,  but  no  Expert  Panel  meetings 
were  held  during  Year  4.  During  Year  5,  all  Expert  Panel  Members  will  be  invited  to  attend  a 
presentation  on  project  findings.  (See  Appendix  E  for  a  listing  of  Expert  Panel  members). 

Task  6.  Project  Planning  Meetings 

The  planning  meetings  outlined  in  Task  6  are  intended  to  ensure  that  both  the  development  of  the 
intervention  and  the  resolution  of  any  problems  that  might  arise  could  be  dealt  with  in  a 
collaborative  fashion  by  VU  and  FHP&R.  Following  the  conclusion  of  site  visits.  Year  4 
teleconferences  were  scheduled  on  an  as-needed  basis,  supplemented  by  frequent  email 
communication.  A  table  of  all  external  meetings  (project  planning  meetings.  Expert  Panel  meetings, 
and  other  assorted  meetings)  is  included  as  Appendix  F.  In  addition  to  these  meetings,  the  VU 
research  team  met  internally  at  least  once  each  week. 

Task  7.  Preparation  of  Final  Reports 

Project  findings  were  presented  to  FHP&R's  Psychological  Health  Forum  on  19-Sept-13,  and 
additionally  presented  at  the  American  Psychological  Association  Annual  Meeting  on  31-JUL-13. 

(See  Appendices  C  and  D).  During  Year  5,  we  anticipate  the  preparation  of  several  manuscripts  for 
submission  to  peer-reviewed  journals  as  well  as  submission  of  the  final  project  report. 


Purdue  University  Scope  of  Work  (SOW)  Tasks 

Task  1.  Analysis  of  merged  Veteran's  Health  Administration  (VHA)  and  DoD  data 

Purdue  was  unable  to  complete  this  task  due  to  the  lack  of  receipt  of  the  VA  data  as  planned. 
Therefore,  VU  requested  a  modification  to  the  SOW  that  was  formally  submitted  on  07-NOV-13 
(see  Appendix  A  for  further  details).  In  brief,  the  SOW  modification  requests  that  Purdue's  SOW  be 
considered  complete  using  DoD  active  duty  data  only. 
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Task  2.  Preparation  of  Final  Reports 

During  Year  4,  Purdue  has  prepared  three  manuscripts,  which  are  in  draft  stage.  The  first  is  an 
overview  of  behavioral  health  symptoms  and  diagnoses  from  PDHA,  PDHRA,  and  health  care 
encounter  data,  encompassing  PTSD,  depression,  and  alcohol  misuse  issues.  The  second  manuscript 
focuses  on  gender  and  race  issues  regarding  PTSD  symptoms  and  diagnoses.  The  third  manuscript 
focuses  on  symptoms  of  traumatic  brain  injury  (TBI)  from  PDHA,  PDHRA,  and  health  care  encounter 
data. 

During  Year  5,  these  manuscripts  will  be  finalized  and  presented  to  FHP&R  and  AFHSC  for  review 
per  agreed  upon  guidelines,  after  which  they  will  be  submitted  to  peer-reviewed  journals.  Purdue 
will  also  participate  in  the  preparation  of  the  final  report. 

KEY  RESEARCH  ACCOMPLISHMENTS 

•  Data  Management  and  Analysis 

o  Secondary  Analysis  (Previous  evaluation  dataset) 

■  Analyses  of  the  previous  evaluation  dataset  have  been  ongoing. 

o  Electronic  Health  Records  Data  (Current  Study)  and  Data  Collected  During  Site 
Visits 

■  Final  data  sets  containing  PDHRA,  PDHA,  and  HCE  data  were  received  from 
AFHSC. 

■  AFHSC  and  site  visit  data  were  cleaned  and  linked  to  create  a  final  data  set 
for  analysis. 

■  Hierarchical  linear  model  regression  analyses  were  used  to  assess  the  impact 
of  the  workshop  on  Service  member  ratings  of  provider  communication 
skills.  Analyses  showed  that  Service  members  rated  provider  communication 
skills  higher  among  workshop  providers  post-workshop  compared  to  before, 
while  no  such  change  was  observed  for  providers  who  did  not  attend  the 
workshop.  (See  Appendices  C  and  D  for  more  detailed  presentation  of 
results). 

■  A  reliable  scale  to  represent  provider  identification  of  Service  member  BH 
concerns  was  created  from  the  Provider  Post-PDHRA  Survey. 

■  Poisson  loglinear  regression  was  used  to  assess  the  impact  of  the  workshop 
on  provider  identification  of  Service  members'  psychosocial  concerns. 
Analysis  showed  that  for  Service  members  who  reported  BH  concerns  on 
VU's  anonymous  survey,  providers  who  attended  the  workshop  showed 
increase  identification  of  BH  concerns  after  the  workshop,  while  no  such 
effect  was  observed  among  non-workshop  providers.  (See  Appendices  C  and 
D  for  more  detailed  presentation  of  results). 

■  Difference  in  Difference  analysis  was  used  to  examine  the  impact  of  the 
workshop  on  provider  documentation  of  concerns  and  referrals  on  PDHRAs. 
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Analysis  showed  that  documentation  of  concerns  was  increased  among 
workshop  providers,  while  medical  referrals  were  decreased  and  Military 
One  Source  referrals  were  increased.  (See  Appendices  C  and  D  for  more 
detailed  presentation  of  results). 

•  Literature  Review.  Relevant  literature  review  has  been  conducted  by  both  VU  and  Purdue, 
which  will  improve  our  ability  to  meet  the  stated  project  aims. 

•  Approvals.  All  IRB  approvals  related  to  the  study  are  in  place,  with  continuing  review 
approvals  received  on  schedule. 

•  Planning  Meetings.  Project  planning  meetings  were  held  as  scheduled  (See  Appendix  F) 

•  Purdue  Items. 

o  Drafted  a  manuscript  that  provides  an  overview  of  behavioral  health  symptoms  and 
diagnoses  from  PDHA,  PDHRA,  and  health  care  encounter  data,  encompassing  PTSD, 
depression,  and  alcohol  misuse  issues. 

o  Drafted  a  manuscript  that  focuses  on  gender  and  race  issues  regarding  PTSD 
symptoms  and  diagnoses. 

o  Drafted  a  manuscript  that  focuses  on  symptoms  of  traumatic  brain  injury  (TBI)  from 
PDHA,  PDHRA,  and  health  care  encounter  data. 


REPORTABLE  OUTCOMES 

During  Year  4,  one  paper  has  been  presented  at  an  academic  conference,  and  a  second  paper  has 
been  presented  to  FHP&R's  Psychological  Health  Forum.  Both  presentations  focused  on  workshop 
outcomes,  with  the  Psychological  Health  Forum  presentation  being  an  expanded  version  of  the 
presentation  given  at  the  American  Psychological  Association's  Annual  Meeting.  Presentation  slides 
for  these  papers  are  included  as  Appendices  C  and  D.  Additional  manuscripts  for  presentation  and 
publication  are  in  the  drafting  stages. 

Douglas,  S.,  Vides  de  Andrade,  A.R.  Boyd,  S.,  Frazer,  N.,  Fraine,  M.,  Webb,  L.,  Bickman,  L.  (July, 
2013).  Communication  Training  for  Heaith  Care  Providers  to  Improve  Miiitary  Mental  Health  Screening. 
Oral  presentation  to  the  American  Psychological  Association  Annual  Meeting,  Honolulu,  Hawaii. 

Douglas,  S.,  Vides  de  Andrade,  A.R.  Boyd,  S.,  Frazer,  N.,  Fraine,  M.,  Webb,  L.,  Bickman,  L. 
(September,  2013).  Communication  Training  for  Heaith  Care  Providers  Improves  Military  Mental  Health 
Screening.  Oral  presentation  to  the  Force  Health  Protection  &  Readiness  Psychological  Health  Forum. 


CONCLUSIONS 

In  Year  4,  VU  built  upon  the  Year  3  analyses  of  workshop  feasibility  and  acceptability  by  conducting 
analyses  on  workshop  efficacy.  Preliminary  analyses  indicate  positive  impact  of  the  workshop:  (a) 
rated  as  acceptable  and  feasible  by  participating  providers;  (b)  increased  providers'  patient- 
centered  communication  skills  and  expected  behaviors  during  PDHRA  interview  as  rated  by  Service 
members;  (c)  increased  providers'  identification  of  BH  concerns  in  context  of  PDHRA  encounters 
where  Service  members  anonymously  reported  BH  concerns;  and  (d)  affected  PDHRA  provider 
documentation,  with  more  concerns  and  One  Source  referrals  yet  fewer  medical  referrals.  During 
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Year  5,  we  will  conduct  further  analysis  relating  to  workshop  efficacy  and  will  prepare  manuscripts 
based  on  our  findings.  We  also  intend  to  consult  with  the  Expert  Panel  in  the  development  of 
targeted  recommendations  based  on  this  work. 
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Appendix  A:  Revised  Scope  of  Work 


(With  Original  Proposal,  2011,  and  2012  Modifications  Appended) 
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IMPROVING  DEPLOYMENT-RELATED  PRIMARY  CARE  PROVIDER  ASSESSMENTS  OF  PTSD  AND 

OTHER  MENTAL  HEALTH  CONDITIONS 
Award  #:  W81XWH-09-2-0172 

Dr.  Susan  D.  Kelley  &  Dr.  Len  Bickman  (Vanderbilt  University) 

Dr.  Sarah  Mustillo  (Purdue  University) 

LCDR  Nicole  Frazer  &  Dr.  Mark  Paris  (FHP&R) 

MODIFIED  SCOPE  OF  WORK  FOR  REVIEW  AND  APPROVAL 

07-NOV-2013 


VII.  STATEMENT  OF  WORK 

A.  Vanderbilt  University  Statement  of  Work 

Extramural  Partner  Site  (no  animal  or  human  use  at  this  site) 

Susan  Douglas  Kelley,  Ph.D.  (PI)  and  Leonard  Bickman,  Ph.D.  (Co-PI) 

Center  for  Evaluation  and  Program  Improvement,  Vanderbilt  University 

Study  Sites 

The  training  and  feedback  intervention  will  be  tested  in  four  to  six  study  sites.  The  number  of  sites  will 
depend  on  staffing  and  PDHRA  throughput.  Site  recruitment  will  be  limited  to  Army  Active  Duty.  Specific 
sites  will  be  determined  in  collaboration  with  the  Expert  Panel  in  the  first  two  months  of  the  project.  An 
estimated  39  providers  in  total  throughout  the  four  to  six  study  sites  will  be  recruited  to  participate  in 
the  study. 


Overall  Project  Timeline 

Proposed  two-year  project  period  from  01  AUGUST  2009  to  31  JULY  2011. 
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-  Analysis  and  preliminary  report 

Aim  1.  PDHRA  secondary  analysis 

-  Data  request 
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-  Preliminary  report 

Aim  2.  Training  &  Feedback 
Intervention  Effectiveness  Study 

-  Development;  recruitment 
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-  Pre-  and  post-audiotape 
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collection 

1 

-  Randomized  study 

■ 

1 

-  Data  request  for  PDFIRA  and 
health  service  utilization  records 

-  Analysis 

Expert  Panel  Meetings 

i 

■ 

■ 

Preparation  of  Final  Reports 

1 

Task  1.  Timing  of  Approvals  and  IRB  (Year  1,  months  1-11) 

If  required  for  this  grant,  VU  and  FHP&R  will  build  on  excellent  working  relationships  based  on  our 
current  research  project.  Appropriate  Army  IRBs  may  include  Regional  Medical  Center  (RMC)  and/or 
installation  IRBs  depending  on  the  study  sites  recruited. 

la.  Multi-project  Institutional  Authorization  Agreement  (lAA)  submitted  to  TMA  Exempt  Determination 
Official  to  establish  the  VU  IRB  as  the  IRB  of  record  (Year  1,  months  1-2) 

lb.  PHDRA  secondary  analysis  protocol  (Year  1,  months  1-4) 

•  Submitted  to  VU  IRB,  estimated  review  time  for  non-human  subjects  protocol  (Year  1,  month  1) 

•  Submitted  to  TMA  Exempt  Determination  Official,  estimated  review  time  (Year  1,  months  3-4) 
fx.  PDHRA  focus  group  protocol  (Year  1,  months  1-4) 

•  Submitted  to  VU  IRB,  estimated  review  time  for  exempt  protocol  (Year  1,  months  1-2) 

•  Submitted  to  appropriate  Army  IRBs  and  estimated  review  time  (Year  1,  months  3-4) 

Id.  Training  and  feedback  intervention  study  protocol  (Year  1,  months  3-10) 

•  Submitted  to  VU  IRB,  estimated  review  time  for  expedited  protocol  (Year  1,  months  3-4) 

•  Submitted  to  appropriate  Army  IRBs,  estimated  review  time  (Year  1,  months  5-11).  Note  that 
final  training  materials  will  be  submitted  for  review  in  months  10-11. 

Task  2.  (Aiml)  PDHRA  focus  groups  (Year  1,  months  1-2,  5-9) 

2a.  Recruitment  of  four  to  six  study  sites  (Year  1,  months  1-2) 

2b.  Development  of  focus  group  protocols  (Year  1,  month  1) 

2c.  Administration  of  2-hour  focus  groups  conducted  at  each  study  site  (Year  1,  months  5-7) 

2d.  Professional  transcription  of  focus  group  audiotapes  ongoing  as  each  completed  (Year  1,  months  5- 

2e.  Qualitative  analysis  will  be  ongoing  as  each  focus  group  completed  with  aggregation  of  findings  after 

all  completed  (Year  1,  months  5-9) 

2f.  Production  of  preliminary  reports  and  briefings  (Year  1,  months  8-9). 

Modified  per  previous  approvais 

Task  3.  (Aim  1)  PDHRA  secondary  analysis  (Year  1,  months  4-9) 

3a.  Data  requests  to  appropriate  information  technology  officer  at  each  Service  for  provider  and  MTF 
identifiers  for  PDFIRAs  completed  between  01/01/06  to  05/51/09  (Year  1,  month  4) 

3b.  Linking  file  created  by  TMA  to  provide  de-identified  dataset  to  VU  containing  non-identifying  SM 
identifier  and  provider/MTF  identifiers  (Year  1,  month  4) 

3c.  Data  management  and  analysis  (Year  1,  months  5-7).  Abbreviated  analytic  timeframe  estimated 
because  we  will  be  adding  this  dataset  to  existing  clean  datasets  with  much  of  the  analytic  programming 
developed. 

3d.  Production  of  preliminary  reports  and  briefings  (Year  1,  months  8-9). 
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Task  4.  (Aim  2)  Training  and  feedback  intervention  effectiveness  study  (Year  1,  months  1-9;  Year  2, 

months  1-11) 

4a.  Recruitment  of  four  to  six  study  sites  (Year  1,  months  1-2) 

4b.  Development  of  training  materials  (Year  1,  months  1-9) 

'1c.  Randomization  of  39  providers  across  four  to  six  study  sites  (Year  1.  month  12) 

'Id.  Collection  of  pre-training  audiotapes  from  39  providers,  consisting  of  one  randomly  selected  hour  of 

PDHRA  interviews  (Year  1,  month  12)  Modified  per  previous  approvais 
^Training  and  feedback  intervention  (Year  2,  months  1-4) 

•  Initial  training  eight-hour  workshop  for  providers  in  the  two  intervention  conditions  (Year  2, 
month  1) 

•  Feedback  through  ongoing  peer  learning  in  treatment  team  format  conducted  at  relevant  study 
sites  for  30-45  minutes  on  a  weekly  or  bi-weekly  schedule  (TBD)  (Year  2,  months  2-4) 

4f.  Measurement  of  implementation  fidelity  and  quality  (Year  2,  months  1-4) 

•  Collection  of  initial  training  workshop  attendance  records,  administration  of  pre-  and  post¬ 
workshop  evaluations  completed  by  attending  providers,  and  audiotaping  of  simulated 
interviews  conducted  by  providers  during  initial  workshop  (Year  2,  month  1) 

•  Collection  of  attendance  records  at  ongoing  treatment  team  sessions  (Year  2,  months  2-4) 

•  Administration  of  post -training  evaluation  survey  to  participating  providers  (Year  2,  month  4) 

4g.  Measurement  of  intervention  outcomes  (Year  2,  months  1-4,  7) 

•  Collection  of  audiotapes  from  39  providers,  consisting  of  one  randomly  selected  hour  of  PDHRA 
interviews,  one  each  month  of  the  study  period  (Year  2,  months  1-4) 

•  Administration  of  SM  satisfaction  survey  for  each  SM  participating  in  a  PDHRA  interview  with 
participating  providers  during  the  study  period  (Year  2,  months  1-4) 

•  Data  requests  to  Army  information  technology  officer  at  each  installation  for  provider  and  MTF 
identifiers  for  PDHRAs  completed  by  participating  providers  during  study  period  (Year  2,  month 
7) 

•  Data  request  to  TMA  for  (1)  de-identified  PDHRAs  completed  during  study  period  for 
participating  providers  during  study  period,  and  (2)  de-identified  health  care  utilization  records 
for  SMs  interviewed  by  participating  providers  for  eight  weeks  post-PDHRA.  Linking  file  will  be 
created  by  TMA  to  provide  de-identified  dataset  to  VU  containing  non-identifying  SM  identifier 
and  provider/MTF  identifiers  (Year  2,  month  7) 

4h.  Data  management  and  analysis  (Year  2,  months  5-11) 

Task  5.  Expert  Panel  meetings  (Year  1,  months  2,  9;  Year  2,  months  2, 11) 

The  development  of  the  intervention  will  guided  by  an  expert  panel  composed  of  SMs  in  leadership 
positions  related  to  the  PDHRA  process  from  all  Services. 

5a.  Four-hour  in-person  meeting  in  Washington,  DC  (Year  1,  month  1;  Year  2,  month  11) 

5b.  Two-hour  teleconference  calls  (Year  1,  month  9;  Year  2,  month  2) 

Task  6.  Project  planning  meetings  (Years  1  and  2,  all  months) 

6a.  Weekly  one-hour  teleconference  calls 

6L  Three  one-day  intensive  project  meetings  to  be  held  at  FHP&R  in  Washington,  DC  (Year  1,  month  1, 
5,  10;  Year  2,  month  2,  6,  11) 

Task  7.  Preparation  of  final  reports  (Year  2,  months  11-12) 

Final  reports  and  briefings  will  be  prepared  according  to  any  guidelines  or  requirements  as  set  forth  by 
the  granting  agency. 
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B.  Purdue  University  Statement  of  Work 

Year  1 

During  the  first  few  months  of  the  first  year  (09/09-11/09),  the  majority  of  the  work  will  be  conducted 
by  personnel  at  the  Roudebush  VA  Medical  Center  in  Indianapolis  and  at  the  Department  of  Defense  in 
Washington,  DC.  The  PI  (Mustillo)  will  establish  data  collection  procedures,  coordinate  data  collection 
between  those  two  sites,  troubleshoot,  and  assist  with  technical  consultation.  During  the  fall  of  2009, 
she  will  make  trips  to  Indianapolis  and  Washington  DC  as  necessary  to  consult  with  project  personnel 
regarding  data  and  data  collection.  In  12/09,  Mustillo  will  obtain  the  data  from  both  the  VA  and*  the 
DoD  and  create  a  dataset  for  the  project.  During  the  spring  (01/10-05/10)  and  summer  (06/10)  Mustillo 
will  clean,  recode,  merge,  and  reshape  data  and  will  begin  data  analyses.  Mustillo  will  also  supervise  a 
research  assistant  to  help  with  these  tasks. 

Year  2 

In  Year  2,  Mustillo  will  continue  to  conduct  data  analyses  in  accordance  with  the  specific  aims  of  the 
project,  with  the  help  of  a  graduate  research  assistant  (08/10-12/10).  Additionally,  Mustillo  and  the 
research  assistant  will  write  a  report  of  the  finding  for  the  Department  of  Defense  as  well  as  manuscripts 
for  publication  in  professional  journals  (01/11-07/11).  Mustillo  will  travel  to  Washington  DC  to  consult 
with  DoD  personnel,  present  key  findings,  and  receive  input  on  analyses  as  necessary.  Mustillo  also  will 
present  at  least  one  manuscript  at  a  research  conference. 

*Added  language 

1.  Suggested  SOW  change:  The  data  sample  for  Purdue  University's  examination  of  the  predictors  of 
positive  PTSD  screens  in  PDHA  and  PDHRA  based  on  the  explanatory  variables  specified  in  the  original 
proposal  has  been  modified  from  active  duty  and  VA  data  to  active  duty  data  only. 

2.  Rationale:  As  of  July,  2012  Sarah  Mustillo  at  Purdue  had  successfully  completed  all  aspects  of  the  sub¬ 
award  scope  of  work  with  non-VA  data  related  to  examining  the  predictors  of  positive  PTSD  screens  in 
PDHA  and  PDHRA  based  on  the  explanatory  variables  specified  in  the  original  proposal.  In  addition, 
Purdue  has  examined  subsequent  PTSD  diagnoses  during  inpatient  and  outpatient  health  care 
encounters.  In  September  of  2012,  Dr.  Mustillo  presented  results  from  one  set  of  analyses  to  the  FHP&R 
Psychological  Health  Forum.  Dr.  Mustillo  also  has  three  articles  in  final  stages,  all  of  which  will  be 
submitted  to  FHP&R  and  AFHSC  for  review  prior  to  submitting  them  for  consideration  in  peer-reviewed 
journals  in  Year  5. 

According  to  information  that  Dr.  Mustillo  has  received  from  the  VA,  Purdue  will  be  unable  to  repeat 
these  analyses  with  the  VA  dataset  as  originally  proposed.  Purdue  has  passed  privacy  approvals  at  both 
the  local  and  national  levels  with  no  problems.  While  there  is  a  completed  Privacy  Review  within  the 
DART,  Purdue  was  informed  that  overall  review  cannot  be  resolved  because  it  was  determined  there 
was  no  legal  authority  for  the  PI  to  send  the  data  to  either  DOD  or  Purdue  University  under  the  Privacy 
Act  of  1974.  The  subjects  of  the  research  are  not  signing  either  a  consent  or  authorization  and  there  is 
no  routine  use  under  34  VA  12.  The  latter  is  the  applicable  SOR  since  the  research  data  is  being  collected 
to  send  it  to  the  other  entities.  The  waiver  of  HIPAA  authorization  does  not  address  the  requirement  in 
the  Privacy  Act,  as  there  is  no  provision  in  the  Act  for  a  waiver  process. 
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These  problems  have  been  duly  reported  in  quarterly  reports  to  USAMRMC-MORP  and  in 
communication  with  our  contacts  at  FHP&R.  Despite  numerous  attempts  to  resolve  these  issues 
together  with  our  cooperative  partners,  there  appears  to  be  no  resolution  to  the  requirement  by  the  VA 
for  informed  consent  by  individual  service  members  because  the  data  being  requested  consist  of  pre¬ 
existing  records. 

Therefore,  as  the  PI  of  the  cooperative  agreement,  VU  proposes  we  modify  Purdue's  SOW  to  reflect  the 
successful  work  already  done  with  the  non-VA  data.  This  is  in  the  best  interests  of  the  overall  project  to 
allow  the  remaining  efforts  in  the  3rd  no-cost  extension  year  to  be  focused  on  the  primary  research 
aims.  There  are  no  changes  to  the  budget. 
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ORIGINAL  PROPOSAL 

IMPROVING  DEPLOYMENT  -  RELATED  PRIMARY  CARE  PROVIDER 
ASSESSMENTS  OF  PTSD  AND  MENTAL  HEALTH  CONDITIONS 
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PROPOSAL  #:  CWS_08_R3_259 

IMPROVING  DEPLOYMENT  -  RELATED  PRIMARY  CARE  PROVIDER 
ASSESSMENTS  OF  PTSD  AND  MENTAL  HEALTH  CONDITIONS 

SUSAN  DOUGLAS  KELLEY,  Ph.D.  (PI)  and  LEONARD  BICKMAN,  Ph.D.  (Co-PI) 

I.  PROJECT  SUMMARY/ABSTRACT 

This  proposal  addresses  the  need  for  research  on  service  delivery  approaches  for 
Service  Members  (SMs)  with  combat-related  physical  or  psychiatric  symptoms,  including  Posttraumatic 
Stress  Disorder  and/or  post-concussive  symptoms.  As  a  primary  care  encounter,  the  post  deployment 
health  reassessment  (PDHRA)  process  is  critical  to  force  health  protection  efforts.  The  proposed  project 
will  develop  and  test  the  effectiveness  of  a  targeted  training  and  feedback  intervention  designed  to  help 
providers  increase  SM  reports  of  behavioral  health  concerns  and  SM  acceptance  of  a  referral  for  further 
assessment.  The  project  builds  on  previous  collaboration  between  Vanderbilt  University  and  Force 
Health  Protection  and  Readiness  and  will  be  applicable  to  all  Service  Branches  and  Components.  We 
propose  two  aims.  (Aim  1)  Development  of  PDHRA-specific  clinical  guidelines  and  training  materials 
through  collaboration  with  key  national  leaders  and  installation-level  stakeholders  involved  in  the 
PDHRA  process,  and  through  a  secondary  analysis  of  PDHRA  data  linked  by  provider.  (Aim  2)  Test  of 
intervention  effectiveness  at  four  to  six  sites  with  39  primary  care  providers  who  conduct  PDHRAs. 
Providers  will  be  randomly  assigned  to  one  of  two  interventions  (training  and  ongoing  feedback  or 
training  only)  or  to  typical  training  (control  group).  Outcomes  include  implementation  fidelity  and 
quality,  content  analysis  of  communication  style  from  interview  audiotapes,  secondary  analysis  of  the 
PDHRA  form  and  SM  health  care  utilization,  and  SM  satisfaction  surveys.  Data  will  be  analyzed  using  a 
longitudinal  repeated  measure  slope-as-outcome  model.  A  secondary  analysis  of  PDHRA  data  will  also 
be  conducted  to  identify  risk  factors  in  the  development  of  Posttraumatic  Stress  Disorder. 

II.  PROJECT  NARRATIVE 

A.  Background  &  Literature  Review 

Post  Deployment  Health  Assessment  (PDHA)  and  Re-Assessment  (PDHRA) 

The  deployment  health  assessment  continuum  includes  a  primary  care  provider  interview,  brief  written 
assessment,  and  health  education  to  the  deployed  force  at  critical  time  points  in  the  deployment  cycle. 
The  health  assessment  process  is  a  therapeutic  encounter  as  well  as  a  health  education  opportunity  that 
is  critical  to  force  health  protection  efforts.  SMs  complete  the  post  deployment  health  assessment 
(PDHA)  at  the  time  of  redeployment  and  the  post  deployment  health  reassessment  (PDHRA)  about  90- 
180  days  later.  Both  include  completion  of  a  self-report  form  and  a  provider  interview.  The  purpose  of 
the  provider  interview  is  to  review  SM  responses  to  the  self-report  questions,  discuss  any  concerns  or 
problems,  provide  education  and  clinical  health  risk  communication  about  common  deployment  health 
concerns,  and  to  make  referrals  for  further  evaluation  as  needed. 

As  a  primary  care  encounter,  the  PDHRA  process  is  a  vital  component  of  the  military  health  care 
system's  attempt  to  care  for  SMs  physical  and  behavioral  health  problems  following  deployment.  It  is 
important  for  primary  care  providers  to  communicate  effectively  with  SMs,  particularly  with  regard  to 
behavioral  health  and  mild  traumatic  brain  injury  (mTBI)  concerns.  The  PDHRA  includes  a  provider 
interview  intended  to  motivate  the  SM  to  disclose  problems  and  agree  to  a  referral.  At  present  the 
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PDHRA  providers  operate  with  little  systematic  information  or  feedback  on  either  the  interview  process 
or  the  outcome  of  a  referral. 

Utility  of  Screening  for  Behavioral  Health  Disorders 

The  utility  of  any  health  risk  appraisal  system  depends  on  the  accuracy  of  the  relevant  screening  tools 
and  procedures.  The  PDHRA  is  considered  a  multi-gate  screening  program.  The  first  gate,  the  SM  self- 
report,  is  a  type  of  threshold-based  screening,  which  typically  consists  of  short  questionnaires 
completed  by  individuals  intended  to  determine  if  they  meet  a  pre-defined  threshold  of  risk  in  need  of 
further  evaluation.  The  second  gate  is  the  clinical  interview,  where  the  health  care  provider  reviews  the 
results  in  conjunction  with  an  interview  to  determine  need  for  further  evaluation  through  referral. 
Accuracy  is  typically  defined  as  the  validity  of  the  screening  tools  and  procedures  as  they  appropriately 
identify  individuals  who  are  or  are  not  at  risk  through  statistical  techniques  such  as  calculation  of 
sensitivity  and  specificity  (discussed  further  as  part  of  Aim  1  in  section  B.3  below).  For  the  PDHRA 
process,  accuracy  can  be  influenced  by  SM  attitudes  toward  reporting  behavioral  health  problems. 

Recent  research  has  demonstrated  that  a  majority  (up  to  60%)  of  military  personnel  who  screen  positive 
for  mental  health  problems  do  not  seek  any  care  (Hoge,  Castro,  Messer,  McGurk,  Cotting,  &  Koffman, 
2004),  a  sobering  result  similar  to  that  found  in  the  general  population  (Willis,  Willis,  Male,  Henderson  & 
Manderscheid,  1998).  Factors  that  influence  attitudes  toward  help  seeking  include  stigma,  anticipated 
utility  of  treatment,  propensity  to  self-disclose,  and  social  support  (Ajzen  &  Fishbein,  1980;  Koenan, 
Goodwin,  Struening,  Heilman,  &  Guardino,  2003;  Salzer  &  Bickman,  1999;  Vogel  &  Wester,  2003;  Vogel, 
Wade,  Wester,  Larson,  &  Hackler,  2007).  Related  barriers  to  help  seeking  behavior,  often  labeled 
organizational  barriers,  that  have  been  identified  by  military  personnel  include:  difficulty  in  getting  time 
off  from  work,  not  having  adequate  transportation,  not  knowing  where  to  go,  difficulty  scheduling 
appointments,  distrust  in  the  mental  health  system,  or  financial  strain  (Chappelle  &  Lumley,  2006;  Hoge 
et  al.,  2004). 

Outreach  and  education  to  reduce  these  perceived  barriers  has  been  suggested  as  a  necessary 
component  of  any  effective  mental  health  care  system  (e.g.,  Kelly  &  Jorm,  2007).  Such  education  may 
include  not  only  'destigmatizing  messages'  and  practical  information  on  how  to  seek  help,  but  also 
information  on  the  effectiveness  of  available  treatments.  It  has  been  suggested  that  greater  confidence 
in  effective  treatment  could  reduce  the  stigma  associated  with  mental  health  disorders  (Corrigan,  2004; 
Meltzer,  Bebbington,  Brugha,  Farrell,  Jenkins,  &  Lewis,  2000;  Sammons,  2005).  A  strength  of  the  PDHRA 
process  is  the  inclusion  of  the  provider  interview,  which  may  serve  as  a  key  factor  in  the  provision  of 
such  education  to  decrease  stigma  and  increase  understanding  of  help-seeking  for  behavioral  health 
problems. 

Clinical  Interview  as  Part  of  Accurate  Health  Risk  Appraisal 

The  PDHRA  process  as  implemented  typically  relies  on  clinical  expertise  to  guide  the  health  risk  appraisal 
process.  The  benefits  of  a  clinical  screening  system  are  the  depth  and  quality  of  assessment  given. 
Clinical  screening  techniques  involve  the  use  of  very  direct  measures  of  behavior  such  as  interview, 
physical  exam,  or  observation.  Moreover,  direct  measurement  often  affords  health  care  providers  the 
opportunity  to  rule  out  false-positives  or  rule  in  false-negatives  that  may  be  due  to  unrelated  factors 
such  as  stigma,  reading  ability,  English  language  proficiency,  etc.  Health  care  providers  also  may  bring  to 
bear  advanced  knowledge  on  the  interpretation  of  the  "clinical  significance"  of  assessment  results  in 
light  of  contextual  variables.  However,  very  little  formal  evaluation  has  been  carried  out  to  assess  the 
relative  accuracy  of  specific  clinical  screening  techniques  such  as  the  clinical  interview  alone  in 
predicting  future  risk..  In  general,  health  services  research  suggests  that  health  care  providers  may  be 
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able  to  identify  individuals  who  are  currently  high  risk,  but  they  are  less  able  to  identify  those  who  are 
going  to  become  high  risk  in  the  future  (Dudley,  1996).  This  is  particularly  relevant  to  the  PDHRA 
process,  given  that  the  purpose  of  the  appraisal  is  to  identify  need  for  further  evaluation  rather  than 
diagnosis. 

Ideally,  all  variation  in  care  would  result  from  differences  in  clinical  features,  patient  preferences,  and 
available  resources  (Eddy,  1984).  In  actual  practice,  clinical  decision  making  does  not  occur  in  a 
consistent  manner  due  to  error  arising  from  several  factors.  Providers  vary  in  their  interview  style  and 
fact-finding  techniques,  which  can  have  a  significant  impact  on  the  nature  of  information  elicited,  and 
subsequent  decisions  (Cox,  Holbrook,  &  Rutter,  1981;  Cox,  Rutter,  &  Holbrook,  1981;  Graham  &  Rutter, 
1968;  Hopkinson,  Cox,  &  Rutter,  1981;  Rutter  &  Cox,  1981;  Rutter,  Cox,  Egert,  Holbrook,  &  Everitt,  1981; 
Rutter  &  Graham,  1968). 

Variability  in  clinical  findings  can  also  result  from  the  experience  and  knowledge  available  to  the  health 
care  provider.  The  initial  hypotheses,  and  often  the  resulting  interpretation  of  presenting  symptoms, 
may  be  influenced  by  such  factors  as  when  and  where  the  health  care  provider  was  trained  and  the 
amount  and  quality  of  the  individual's  clinical  experience.  Recent  research  has  suggested  that  among 
expert  health  care  providers,  the  decision-making  process  is  typically  one  of  pattern  recognition  or 
direct  automatic  retrieval  of  facts  relevant  to  interpreting  the  meaning  of  symptoms  presented  by  the 
individual  being  assessed  (Elstein  &  Schwarz,  2002).  Studies  of  physician  behavior  conducted  by  Patel, 
Arocha,  Diermeier,  How,  &  Mottur-Pilson  (2001)  have  demonstrated  that  experts  are  better  able  to 
organize  information  into  manageable  and  meaningful  "chunks",  are  less  likely  to  attend  to  irrelevant 
information,  and  more  likely  to  generate  logical  hypotheses  based  on  the  data  presented  (Patel  & 

Groen,  1991).  Because  experts  make  greater  use  of  clinical  "schemas"  or  prototypes  of  typical  cases, 
they  are  able  to  more  efficiently  and  effectively  integrate  relevant  sources  of  information  during  the 
interview  and  decision-making  process  (Patel  et  al.  2001). 

Both  idiosyncratic  and  common  cognitive  errors  in  social  information  processing  are  possible  sources  of 
bias  in  clinical  interpretation  and  decision  making.  For  example,  health  care  providers  may  hold 
idiosyncratic  beliefs  or  assumptions  based  on  their  observed  behaviors  and  characteristics  (Van  Ryn, 
2002).  Van  Ryn  and  Burke  (2000)  found  that  patient  race  and  socio-economic  status  (SES)  were 
associated  with  several  health  care  provider  perceptions  regarding  intelligence,  personality,  risk 
behaviors,  and  compliance  with  medical  advice.  Caucasians  were  about  twice  as  likely  as  African 
Americans  to  be  rated  as  at  no  risk  for  substance  use  and  noncompliance.  Patients  in  the  lowest  SES 
category  were  twice  as  likely  to  be  rated  as  irresponsible  and  irrational  compared  with  patients  in  the 
middle  and  upper  SES  categories  (Van  Ryn  &  Burke,  2000).  Health  care  providers  holding  these  beliefs 
may  feel  that  a  person  is  less  deserving  of  treatment  based  on  certain  social  or  behavioral  characteristics 
(Van  Ryn,  2002).  In  turn,  perceived  stereotyping  by  health  care  providers  may  affect  patient  attitudes 
and  perceptions  and  interactions  between  the  patient  and  health  care  provider  (U.S.  Department  of 
Health  and  Human  Services  [USDHHS],  2008;  Van  Ryn,  2002). 

Although  the  decisions  of  some  health  care  providers  may  be  influenced  by  assumptions  related  to 
socio-cultural  background  of  patients  interviewed,  all  health  care  providers  are  vulnerable  to  a  set  of 
predictable  set  of  cognitive  errors,  biases,  and  heuristics.  These  errors  are  due  in  large  part  to  the 
cognitive  demands  placed  on  the  health  care  provider  during  the  decision-making  process  during  which 
he  or  she  must  apply  "heuristics"  or  cognitive  shortcuts  to  make  quick  sense  of  the  large  amount  of 
information  they  are  presented  with.  Meehl  (1954)  was  the  first  to  make  a  distinction  between  clinical 
and  actuarial  (also  known  as  statistical,  mechanical,  or  algorithmic)  decision-making  from  an  information 
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processing  perspective.  In  this  context,  clinical  decision  making  is  defined  as  the  internal  process  of 
combining  information  in  order  to  make  a  treatment  decision,  whereas  actuarial  decision  making  is  the 
process  of  making  conclusions  on  the  basis  of  established  relationships  between  the  data  and  condition 
of  interest  (Dawes,  Faust,  &  Meehl,  1989). 

Research  over  the  last  70  years  has  found  that  actuarial  decision  making  is  more  accurate  and  less 
variable  than  clinical  decision  making  in  most  cases.  In  a  review  of  617  comparisons  in  136  studies 
published  between  1920  and  1994,  Grove,  Zald,  Lewbow,  Snitz,  and  Nelson  (2000)  found  only  eight 
studies  in  which  clinical  decision  making  surpassed  the  accuracy  of  actuarial  decision  making.  Several 
factors  have  been  cited  as  possible  explanations  for  the  superiority  of  actuarial  over  clinical  decision 
making.  First  and  foremost,  the  human  brain  is  not  efficient  at  noticing,  selecting,  categorizing, 
retaining,  retrieving,  manipulating,  and  appropriately  applying  information  for  the  purpose  of  making 
inferences  (Grove  &  Meehl,  1996).  As  a  result,  clinical  decision  making  is  prone  to  fluctuations  in 
judgment  due  to  the  influence  of  cognitive  errors,  application  of  heuristics,  and  biases  (Dawes  et  al., 
1989). 

Findings  from  Collaborative  VU-FHP&R  Project 

As  part  of  their  current  contract  with  DoD  (see  section  B.5  below  for  further  detail),  VU  has  conducted 
preliminary  statistical  analysis  of  a  de-identified  secondary  dataset  representing  available  PDFIRA 
records  for  all  SMs  between  the  dates  of  01/01/06  and  04/30/08  (N=595,191).  A  subset  of  359,387 
records  were  used  for  analysis  representing  SMs  deployed  to  OIF/OEF  once  during  the  time  period  who 
were  in  reserve  or  active  components  of  the  Army,  Navy,  Air  Force,  or  Marine  Corps  at  the  time  of  the 
PDFIRA.  A  main  focus  of  the  secondary  analysis  was  to  examine  the  relationship  between  a  SM's  self 
report  and  their  clinician's  report  of  concerns  and  offer  of  referrals.  Referral  was  defined  as  any  referral 
made  to  medical  (i.e.,  primary  care,  specialty  care)  or  non-medical  care  (i.e.,  case  manager,  family 
support). 

The  correlation  between  SM  self-report  of  problems  and  provider-report  of  concerns  was  moderate  (r  = 
0.64).  This  suggests  that  41%  of  the  variance  in  provider  concern  count  comes  from  the  SM's  self-report 
of  problems,  with  59%  from  other  sources.  While  one  source  of  the  difference  is  unreliability,  another 
might  be  "value  added"  by  the  provider  (observing  problems  that  the  SM  did  not  self-report  or 
identifying  previously  reported  problems  as  no  longer  of  concern).  This  indicates  the  necessity  of 
understanding  how  the  clinical  decision-making  process  contributes  to  PDFIRA  outcomes. 

Other  analyses  suggest  a  need  to  understand  how  providers  view  different  types  of  SM  self-reported 
symptoms.  For  example,  we  have  found  the  relationship  between  the  probability  of  any  referral  and  the 
number  of  self-reported  symptoms  varies  by  the  type  of  symptoms.  For  physical  health  symptoms,  the 
greater  the  number  of  symptoms,  the  greater  the  likelihood  of  referral.  This  is  not  the  case  with  self- 
reported  exposure  concerns.  SMs  could  endorse  23  separate  persistent  major  concerns  related  to  the 
health  effects  of  various  exposures  or  encounters  during  deployment.  These  ranged  from  'DEBT  insect 
repellent  applied  to  skin'  to  'depleted  uranium.'  About  50%  of  SMs  received  any  referral  with  two 
concerns  endorsed,  but  there  was  a  relatively  low  cumulative  probability  of  referral  for  additional 
exposure  concerns  reported.  For  behavioral  health  problems  (symptoms  related  to  PTSD,  depression, 
alcohol  problems,  and  relationship  conflicts)  almost  one-fifth  of  SMs  did  not  receive  a  referral  even 
though  all  questions  (9)  were  endorsed.  These  findings,  while  preliminary,  indicate  that  there  is  a  clear 
need  to  better  understand  how  the  clinical  decision-making  process  contributes  to  PDFIRA  outcomes. 

Need  for  Training  and  Feedback  Interventions 
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The  judgment  of  even  the  most  experienced  health  care  provider  can  be  compromised  by  incomplete  or 
inaccurate  information  on  the  causes,  natural  course,  progression,  and  most  reliable  predictors  of  risk 
across  the  life-span.  Although  access  to  up-to-date  information  and  professional  development  for  less 
experienced  health  care  providers  may  appear  to  be  a  logical  response  for  dealing  with  variation  in 
levels  of  training  and  experience,  the  development  of  expertise  in  clinical  assessment  is  not  necessarily 
linear.  For  example,  Patel  et  al.  (2001)  made  an  important  distinction  between  the  performance  of 
experts  (experts  working  on  tasks  relevant  to  their  primary  domain  of  expertise),  sub-experts  (experts 
working  outside  of  their  primary  domain  of  expertise),  intermediates  (individuals  with  skills  at  an 
intermediate-stage  between  expert  and  novice  such  as  intern  or  resident  health  care  providers),  and 
novices  (individuals  with  limited  experience  and  content  knowledge).  Interestingly,  research  has 
demonstrated  that  intermediates  may  perform  more  poorly  than  novices  on  specific  tasks  such  as  the 
recall  of  patient  data  (Patel  &  Groen,  1991),  explanation  of  clinical  problems  (Patel,  Groen,  &  Scott, 

1988)  and  generation  of  well-formed  diagnostic  hypotheses  (Arocha,  Patel,  &  Patel,  1993).  This 
unexpected  "intermediate  effect"  has  been  explained  as  a  consequence  of  the  natural  ebb-and  flow  of 
human  learning  and  development.  Specifically,  it  has  been  hypothesized  that  the  development  of 
expertise  involves  a  continuous  process  of  learning,  re-learning,  and  application  of  new  knowledge 
during  which  there  are  periods  of  apparent  decreases  in  mastery  and  performance  among  intermediates 
as  new  information  is  learned  and  integrated  (Patel  et  al.,  2001).  As  such,  the  potential  value  of 
additional  training,  feedback  and  professional  development  may  depend  on  the  health  care  provider's 
particular  stage  of  development  with  regard  to  his/her  level  of  expertise  in  their  particular  work  context. 

Further,  a  health  care  provider's  clinical  experience  does  not  come  from  a  truly  representative  sample  of 
the  population.  As  a  result,  his  or  her  perception  of  the  relationship  between  variables  is  not  necessarily 
representative  (Dawes  et  al.,  1989).  Consequentially,  health  care  providers  routinely  ignore  base  rates 
when  estimating  the  probability  of  a  given  diagnosis.  Instead,  they  may  consider  each  hypothetical 
diagnosis  equally  likely  because  they  are  looking  at  how  close  a  particular  case  is  to  a  diagnostic 
category  or  previously  seen  cases  (also  known  as  the  representativeness  heuristic;  Elstein  &  Schwarz, 
2002).  There  is  also  a  tendency  to  overestimate  the  frequency  of  unusual  and  easily  recalled  events. 
Flence,  health  care  providers  tend  to  overemphasize  rare  conditions  when  making  clinical  judgments 
(also  known  as  the  availability  heuristic;  Elstein  &  Schwarz,  2002). 

In  addition,  over  the  course  of  their  training  and  clinical  practice,  many  health  care  providers  may 
develop  inaccurate  beliefs  about  the  association  between  risk  factors  and  observed  symptoms  (Dawes 
et  al.,  1989).  Unfortunately,  these  incorrect  assumptions  may  be  used  as  a  frame  to  guide  the  clinical 
fact-finding  process  itself.  As  a  result  of  the  confirmation  bias,  health  care  providers  may  be  more  likely 
to  attend  to  information  that  supports  this  initial  hypothesis  and  reject  or  downplay  evidence  that 
refutes  it  (Dawes  et  al.,  1989).  Error  resulting  from  the  interplay  of  these  processes  is  compounded  by 
the  fact  that  past  predictions  are  generally  recalled  as  being  more  accurate  than  they  were  (also  known 
as  hindsight-bias),  thus  inflating  the  health  care  provider's  assessment  of  his  or  her  actual  decision 
making  ability  (Dawes  et  al.,  1989)  and  decreasing  the  likelihood  they  will  regularly  pursue  alternate 
clinical  hypotheses  for  presenting  symptoms.  In  addition  to  these  common  cognitive  errors,  the  personal 
regret  a  given  health  care  provider  anticipates  feeling  if  he/she  rendered  an  incorrect  diagnosis  or  the 
patient  was  not  provided  appropriate  care  may  also  influence  clinical  decision-making  and  referral 
practices  in  rather  unpredictable  ways.  Given  the  robustness  of  previous  research  on  the  identification 
of  problems  in  clinical  decision  making  and  its  relationship  to  the  improving  the  PDFIRA  interview 
process  it  should  be  evident  that  research  on  how  to  improve  the  PDFIRA  should  be  a  high  priority. 

B.  Research  Design  &  Methods 
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B.l  General  Overview 
Specific  Aims 

This  proposal  is  written  in  response  to  the  Medical  Research  and  Materiel  Command  (MRMC)  intramural 
Department  of  Defense  (DoD)  FY2008  War  Supplemental  Intramural  Announcement.  The  proposal 
specifically  addresses  the  need  for  research  on  the  promotion  of  treatment  interventions,  prevention 
strategies,  and  service  delivery  approaches  targeting  Service  members  (SMs)  with  combat-related 
Posttraumatic  Stress  Disorder  (PTSD)  and/or  psychiatric  and  related  physical  and/or  post-concussive 
conditions. 

More  specifically,  it  addresses  three  areas  of  special  consideration  when  developing  evidence-based 
treatment  interventions  for  combat-related  PTSD  and  co-occurring  problems: 

•  Approaches  in  primary  care  to  reduce  physical  and  psychiatric  co-morbidity  associated  with 
PTSD  through  increased  effectiveness  of  early  screening  and  identification  leading  to  increased 
SM  access  to  and  use  of  appropriate  care 

•  Evidence-based  strategies  to  enhance  engagement,  retention,  compliance,  and  return  to  duty 
performance  of  SMs  specificaliy  targeting  engagement  in  the  assessment  process  through 
evidence-based  strategies  to  enhance  provider's  interview  and  clinicai  communication  practices 
during  deployment  reiated  assessments.  SM  engagement  includes  both  elicitation  of  reports  of 
behavioral  health  concerns  and  motivation  to  accept  referrals  for  further  behavioral  health 
evaluation  and/or  treatment 

•  Clinical  competencies  and  treatment  fidelity  guidelines  through  the  development  and  testing  of 
targeted  training  and  ongoing  feedback  reiated  to  the  primary  care  provider's  interview  and 
clinical  communication  practices  during  depioyment  related  assessments 

The  primary  focus  of  the  proposed  project  is  to  determine  the  impact  of  a  targeted  training  and 
feedback  intervention  related  to  the  primary  care  provider's  interview  and  clinical  communication 
practices  during  deployment  related  assessments  on  eliciting  SM  reports  of  behavioral  health  concerns 
and  in  motivating  SM  acceptance  of  a  referral  for  further  behavioral  health  evaluation  and/or 
treatment.  The  development  of  the  intervention  is  informed  by  substantial  previous  collaboration  by 
Vanderbilt  University  (VU)  and  FHP&R  that  identifies  clinician  communication  patterns  and  behaviors 
that  contribute  to  SM  problem-reporting  and  extensive  analysis  of  a  large  PDFIRA  dataset.  The 
intervention  development  will  be  applicable  to  all  Service  Branches  and  Components.  Specific  aims 
include: 

Aim  1;  Determine  key  elements  of  and  current  impact  of  training  programs  for  deployment  related 
assessments.  The  focus  will  be  on  guidance  related  to  eliciting  candid  reporting  of  behavioral  health 
concerns,  identification  of  behavioral  health  concerns  that  warrant  referral  and  motivating  the  SM  to 
accept  a  referral  for  further  evaluation  and/or  treatment  for  behavioral  health  conditions  and  concerns. 
This  aim  will  be  accomplished  through  (a)  Expert  Panel  review  of  results  from  current  2007-2009  VU- 
FFIP&R  collaboration  to  determine  criteria  for  clinical  competencies;  (b),  process-action  research 
techniques,  including  focus  groups  with  key  stakeholders  involved  in  the  PDHRA  process,  to  assist  in 
identifying  key  elements  for  training  interventions  relevant  to  content,  format,  and  implementation 
and,(c)  secondary  analysis  of  PDFIRA  data  from  a  specifically  developed  database  that  includes  provider 
and  MTF  identifiers  will  allow  identification  of  variability  in  concerns  and  referrals  attributed  to  the 
provider,  over  and  above  SM  self-reported  problems.  The  results  of  the  secondary  analysis  will 
complement  the  results  of  a  2007-2009  contract  that  includes  several  methods  to  determine  factors 
that  influence  appropriateness  of  referral  through  the  PDFIRA.  This  phase  of  research  will  result  in  the 
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development  of  a  training  and  feedback  intervention  intended  to  improve  the  primary  care  provider's 
interview  and  clinical  communication  practices  during  deployment  related  assessments.  This  aim 
includes  data  from  all  Service  Branches  and  Components.  In  addition,  the  Expert  Panel  will  be  comprised 
of  members  from  the  PDHRA  Working  Group,  which  consists  of  leaders  from  all  services. 

Aim  2:  Evaluate  the  effectiveness  of  a  targeted  training  and  feedback  program  on  primary  care 
provider's  interview  and  clinical  communication  patterns  related  to  SM  behavioral  health  condition 
identification  and  referrals.  Based  on  the  findings  from  Aim  1,  a  feedback  and  training  intervention  for 
providers  will  be  tested  with  a  group  of  39  primary  care  providers  who  conduct  deployment-related 
assessments  in  four  to  six  sites.  Providers  will  be  randomly  assigned  to  one  of  two  interventions  (training 
and  ongoing  feedback  or  training  only)  or  to  whatever  typical  training  they  receive.  This  latter  arm  will 
serve  as  the  control  group.  Implementation  will  be  measured  through  training  attendance  records, 
evaluations  completed  by  the  providers,  and  review  of  taped  simulated  interviews  as  a  formative 
assessment.  Outcome  measurement  will  include  pre-and  post-training  content  analysis  of 
communication  style  from  audiotapes  and  an  analysis  of  secondary  data  (including  electronic  records  for 
PDHRA  and  health  care  utilization).  The  latter  will  include  an  analysis  of  concordance  between  SM  self- 
reported  problems  and  provider-reported  concerns;  referral  characteristics;  SM  acceptance  of  referral; 
concordance  between  PDHRA  referral  characteristics  and  subsequent  health  care  utilization  In  addition 
data  will  be  obtained  from  a  brief  post-interview  satisfaction  survey  completed  by  the  SM. 

B.2  Subject  Recruitment 

The  primary  care  providers  are  considered  to  be  the  subjects  for  the  proposed  project.  The  feedback 
and  training  intervention  is  not  intended  to  affect  SMs  directly,  but  rather  indirectly  by  influencing 
clinician  behavior  change  through  training  and  feedback  on  deployment-related  assessments.  Due  to  the 
layered  nature  of  the  project  design,  participants  and  recruitment  methods  are  discussed  by  aim  and 
purpose.  If  funded,  the  PI  will  ensure  that  all  study  procedures  and  tools  are  reviewed  by  the 
appropriate  institutional  review  boards. 

Aim  1:  PDHRA  Focus  Groups 

A  group  of  six  to  eight  staff  associated  with  the  PDHRA  process  at  each  of  the  four  to  six  study  sites  will 
be  invited  to  participate  in  a  2  hour  focus  group  for  purposes  of  identifying  key  elements  for  a  feedback 
and  training  intervention  relevant  to  content,  format,  and  implementation.  Targeted  participants  will 
include  primary  care  providers,  program  managers,  case/referral  managers,  commanders/unit  leaders, 
and  SMs  who  have  recently  been  through  the  PDHRA  process.  FHP&R  key  personnel  will  identify  and 
approach  potential  participants.  Since  no  client-level  data  will  be  discussed  and  all  data  collected  focus 
on  formative  feedback  of  intervention  tools  and  procedures,  it  is  anticipated  that  this  sub-study  will  be 
exempt  from  IRB  requirements.  To  facilitate  ease  of  participation,  the  project  team  will  travel  to  each  of 
the  sites.  Food  and  beverages  will  be  provided  at  each  location. 

Aim  1:  Secondary  Analysis 

The  data  to  be  analyzed  in  the  secondary  analysis  will  consist  of  existing  medical  health  records  related 
to  the  PDHRA  process  (DD  Form  2900)  and  subsequent  health  care  utilization  following  the  PDHRA. 
Provider  and  MTF  identification  will  be  provided  by  individual  Service  medical  record  systems  (e.g., 
AHLTA).  Prior  to  receipt  by  VU,  all  SM  identifying  information  will  be  removed.  Since  no  identifying 
information  is  involved  related  to  SMs,  it  is  anticipated  that  sub-studies  will  be  considered  non-human 
subjects. 

Aim  2:  Intervention  Study 
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The  feedback  and  training  intervention  will  be  tested  in  four  to  six  project  sites,  with  a  total  of  39 
providers  needed  to  provide  adequate  statistical  power  for  the  study.  The  number  of  sites  will  depend 
on  staffing  and  PDHRA  throughput.  Site  recruitment  will  be  limited  to  Army  Active  Duty  to  decrease  the 
potential  influence  of  cross-service  differences  in  medical  facilities  and  SM  deployment  patterns. 
Installations  will  be  approached  and  recruited  by  the  project  team  in  collaboration  with  relevant 
individuals  from  the  Expert  Panel.  Once  the  sites  are  determined,  all  providers  from  each  site  will  be 
given  a  description  of  the  study  before  being  asked  to  volunteer.  The  description  will  include  an 
overview  of  the  study  purpose,  time  commitment  and  schedule  of  activities,  and  measurement 
procedures.  As  part  of  participation  in  the  study,  training  attendance  and  evaluations  of  training  will  be 
administered  to  providers.  Simulated  interviews  conducted  during  the  initial  training  workshop  as 
formative  assessments  will  be  audio  taped  and  later  content  coded.  All  participating  providers  will  be 
asked  to  audiotape  a  randomly  selected  hour  of  interviews  prior  to  the  initial  training,  and  four  more 
randomly  selected  hours  over  the  course  of  the  study  (one  per  month).  Providers  participating  in  the 
training  and  feedback  experimental  condition  will  also  be  asked  to  use  taped  interviews  throughout  the 
study  as  part  of  ongoing  training  and  feedback.  Audio  recorders  will  be  shipped  directly  to  providers  for 
each  taping,  then  shipped  by  them  to  FHP&R  in  SASE  envelopes.  All  audiotapes  will  have  the  SM  de- 
identified  by  FHP&R  prior  to  receipt  at  VU.  Informed  consent  procedures  will  be  used  for  audio 
recording  of  individual  interviews.  Any  measurement  procedures  that  involve  individual  SMs  will  either 
be  anonymous  or  de-identified  prior  to  data  receipt  by  VU.  The  brief  SM  satisfaction  survey  will  be 
completed  by  each  SM  following  the  PDHRA  interview  with  a  participating  provider.  SMs  will  be  asked  to 
note  the  date  and  time  of  the  interview  and  the  provider  who  conducted  the  interview,  but  otherwise 
no  other  identifying  information  will  be  requested  from  the  SM.  These  surveys  will  be  collected  by  the 
installation  PDHRA  program  manager  and  shipped  to  VU.  De-identified  electronic  medical  records  for 
SM  PDHRAs  and  eight  weeks  subsequent  health  care  utilization  will  be  provided  to  VU.  It  is  anticipated 
that  this  sub-study  will  undergo  expedited  institutional  review  as  all  individual  SM  data  are  de-identified 
prior  to  receipt  by  VU  and  the  focus  is  on  standard  of  care  procedures.  Three  months  after  the  study 
period  is  complete,  electronic  records  for  PDHRA  and  health  care  utilization  will  be  requested  for  all  SMs 
interviewed  by  participating  providers.  These  will  be  de-identified  prior  to  receipt  by  VU.  FHP&R  will 
maintain  a  linking  file  that  will  be  used  to  match  relevant  sources  of  data  at  the  SM  level  (audiotapes 
and  electronic  records). 

B.3  Procedures,  Analysis,  and  Interpretation  by  Research  Aim 

Aim  1:  Determine  key  elements  of  and  current  impact  of  training  programs  for  deployment  related 
assessments. 

The  development  of  the  training  and  feedback  intervention  will  follow  an  iterative  process  of 
development  and  improvement.  Complementary  procedures  will  be  used  involving  several  key 
stakeholders  including  national  leaders  in  the  PDHRA  process  through  the  Expert  Panel,  and  individuals 
at  the  installation  level  through  on-site  focus  groups.  A  strength  of  the  proposed  project  is  the  existing 
foundation  of  current  research  and  excellent  working  relationships  with  individuals  at  national  and  local 
levels.  To  guide  the  development  process,  weekly  project  team  meetings  will  be  held  for  the  purposes  of 
planning,  discussing  progress,  and  problem-solving.  These  joint  VU-FHP&R  meetings  will  be  held  by 
teleconference  call  with  three  project  meetings  each  year  in  DC.  In  addition,  two  meetings  will  be  held 
each  year  with  the  Expert  Panel  as  described  previously  in  Section  B.2,  to  ensure  that  intervention 
development  is  fully  informed  by  needs  and  resources  of  all  Service  Branches  and  Components. 

The  four  primary  activities  for  Aim  1  are  described  in  detail  below: 

•  Aim  l.a.  Expert  Panel  review  of  results  from  current  2007-2009  VU-FHP&R  collaboration  to 
determine  criteria  for  clinical  competencies 

•  Aim  l.b.  Focus  groups  with  key  stakeholders  involved  in  the  PDHRA  process 
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•  Aim  l.c.  Secondary  analysis  of  PDHRA  data  to  identify  variability  attributed  to  the  provider 

•  Aim  l.d.  Secondary  analysis  of  PDHRA  data  to  identify  variables  associated  with  the 
development  of  PTSD. 

Aim  l.a.  Expert  Panel  review  of  results  from  current  2007-2009  VU-FHP&R  collaboration  to  determine 

criteria  for  clinical  competencies. 

The  Expert  Panel  currently  reviews  all  planning  and  findings  resulting  from  the  current  2007-2009  VU- 
FHP&R  collaboration.  Of  note  for  establishing  clinical  competencies,  VU  is  currently  in  the  process  of 
analyzing  a  de-identified  sample  of  300  audio  taped  telephone  interviews  conducted  as  standard 
operating  procedure  for  the  PDHRA  process  for  SMs  in  the  Reserve  and  National  Guard.  Typically,  the 
SM  completes  the  self-report  section  of  the  PDHRA  online,  and  then  calls  the  provider  organization  to 
speak  with  a  health  care  provider  who  has  access  to  the  SMs  PDHRA  form  online  and  completes  the 
provider  portion  of  the  PDHRA  screening.  The  goal  of  this  current  sub-study  is  to  evaluate  the 
effectiveness  of  provider  communication  and  interviewing  skills  in  eliciting  SM  reports  of  problems, 
particularly  focusing  on  sensitive  issues  such  as  behavioral  health  problems.  VU  is  content  coding  the 
nature  and  quality  of  the  socio-emotional  exchange  and  task-oriented  exchange  between  SMs  and 
clinical  interviewers  recorded  in  the  electronic  audio  files.  The  coding  scheme  is  briefly  reviewed  here, 
although  it  should  be  noted  that  this  sub-study  is  currently  underway  under  existing  funding. 

The  expert  panel  will  meet  twice  in  person  and  twice  by  teleconference  throughout  the  project  with  the 
following  objectives  in  mind: 


Schedule  and  Purpose  of  Expert  Panel  Meetings 


Time 

Format 

Purpose 

Year  1,  Month 

1 

In  person 
(4  hours) 

Project  planning  including  preparation  for  site  recruitment; 
review  of  project  methods  and  procedures;  review  of 
intervention  development  plan 

Year  1,  Month 

9 

Telcon  (2 
hour) 

Review  preliminary  intervention  manual  and  support 
procedures;  discuss  modifications  based  on  Aim  1  preliminary 
results 

Year  2,  Month 

2 

Telcon  (2 
hour) 

Review  implementation  of  initial  training  and  project  status  for 
all  sites  participating  in  study 

Year  2,  Month 
11 

In  person 
(1  day) 

Review  preliminary  study  findings;  discuss  intervention 
modifications  based  on  Aim  2  results;  focus  on  cross-Service  and 
Branch  needs  and  resources  to  enhance  broad-scale  roll  out 

The  Expert  Panel  will  be  convened  twice  in  person  in  DC  (at  the  beginning  and  end  of  the  project)  and 
twice  by  telephone  conference  call  to  provide  overall  guidance  to  the  project,  with  a  particular  emphasis 
on  cross-service  applications.  Each  in-person  and  telephone  conference  will  include  large-group  review 
and  discussion.  In  addition,  the  in-person  conferences  will  include  small  working  groups  to  provide 
guidance  and  recommendations  in  specific  areas  (e.g.,  adaptations  to  interview  guidelines  for  civilian  or 
military  providers). 

Coding  of  SM-Provider  Socio-emotional  and  Task-Oriented  Exchanges.  The  content  coding 
system  is  called  the  Roter  Interaction  Analysis  System  (RIAS).  It  is  an  internationally  recognized 
instrument  that  has  emerged  as  the  most  widely  used  system  for  coding  communication  in  medical 
encounters.  The  RIAS  (http://www.rias.org)  was  derived  from  social  exchange  theories  related  to 
interpersonal  influence,  problem  solving,  and  reciprocity  and  can  be  used  with  audiotapes  or 
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videotapes.  Roter  and  Larson  (2002)  provide  detailed  information  regarding  the:  (a)  practicality,  (b) 
functional  specificity,  (c)  reliability,  and  (d)  predictive  validity  of  the  RIAS,  so  this  information  is  not 
discussed  in  detail  here.  For  example,  Roter  and  Larson  (2002)  report  that  RIAS  reliability  averages  .85 
for  both  patient  and  physician  categories  based  upon  the  Pearson  correlation  coefficient  (r),  and  this 
reliability  remains  in  the  acceptable  range  after  instrument  translation  to  many  European  languages. 

The  RIAS  focuses  on  two  broad  categories  of  communication  patterns:  (a)  socio-emotional  exchange  and 
(b)  task-focused  exchange  (similar  to  Gallagher,  Hartung,  &  Gregory's  2001  approach).  Within  each  of 
the  two  categories,  numerous  codes  can  be  used  to  identify  the  nature  of  information  given  and 
received  as  well  as  the  general  tone  of  the  medical  encounter.  For  example,  the  RIAS  encodes  provider 
empathy,  level  of  psychosocial  talk,  turn-taking,  affirmation/listening  statements,  and  participatory 
decision  making.  The  manual  also  provides  detailed  examples  and  coding  instructions.  Using  the  RIAS, 
utterances  or  thought  units  (information  events)  (Waitzkin,  1985)  are  analyzed  as  the  smallest  units  of 
verbal  communication  patterns;  and  coding  is  done  directly  from  recording  media  (electronic  audio  files 
in  this  case)  eliminating  the  transcription  step  and  allowing  for  incorporation  of  voice  tone  and  phrasing 
cues  in  assigning  appropriate  codes.  The  RIAS  manual  provides  instructions  about  how  to  code  all 
utterances  or  information  events  on  an  audio  or  videotape  into  39  mutually  exclusive  categories. 
Criticisms  of  the  RIAS  include  issues  such  as:  (a)  not  coding  sequences  within  topics,  (b)  not  coding 
patient's  signals  of  interest/attentiveness,  and  (c)  not  coding  interruptive  speech  (see  Roter  &  Larson, 
2002;  Sandvik,  Eide,  Lind,  Graugaard,  Torper,  &  Finset,  2002).  These  limitations  will  not  hinder  the 
proposed  research. 

Determining  Criteria  for  Clinical  Competencies.  These  data  provide  a  rich  window  into  the 
clinical  interview,  vital  to  making  any  specific  determination  of  clinical  competencies  for  PDFIRA 
communication  and  interviewing  skills  for  health  care  providers.  The  results  will  be  reviewed  by  the 
project  team  and  the  Expert  Panel  with  recommendations  made  for  developing  clinical  competencies  for 
the  training  and  feedback  intervention.  These  will  guide  the  development  of  relevant  training  materials 
and  the  training  model  overall,  described  further  in  Aim  2  below. 

Aim  l.b.  Focus  groups  with  key  stakeholders  involved  in  the  PDFIRA  process. 

In  order  to  obtain  user  perspectives  to  guide  development,  a  focus  group  will  be  held  with  key 
stakeholders  at  each  of  the  sites  where  the  intervention  effectiveness  study  is  to  be  conducted.  Key 
stakeholders  are  those  involved  with  the  PDFIRA  process,  including  primary  care  providers,  program 
managers,  case/referral  managers,  commanders/unit  leaders,  and  SMs  who  have  recently  been  through 
the  PDFIRA  process. 

The  primary  goal  of  the  focus  groups  is  to  identify  key  elements  for  a  training  and  feedback  intervention 
relevant  to  content,  format,  and  implementation.  Presented  seminar-style,  several  methods  will  be  used 
to  enhance  clinical  application  of  information  and  data  collection.  Following  a  brief  introduction  of 
purpose  of  the  meeting  and  the  proposed  intervention,  case  example  vignettes  will  be  used  to  guide 
discussion  of  training  and  use  of  feedback  (Veloski  et  al.,  2005).  Participants  will  take  part  in  discussion 
groups  that  measure  acceptance,  perceived  helpfulness,  and  suggestions  for  improvement  to  training 
and  feedback.  Discussion  will  focus  not  only  on  the  PDFIRA  process  generally,  but  specifically  on  how  to 
tailor  the  intervention  to  local  needs  and  resources.  These  will  include  typical  staffing  patterns;  current 
quality  improvement  initiatives  related  to  the  PDFIRA;  supervisory  and  case  review  structures;  and 
training  resources,  including  structures  to  support  staff  attendance.  The  focus  groups  will  be  conducted 
following  a  structured  protocol,  which  will  be  developed  in  the  early  phase  of  this  study.  All  focus  groups 
will  be  audio-recorded  and  transcribed  by  a  professional  transcriber.  Data  will  be  analyzed  using 
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standard  qualitative  analysis  procedures  (e.g.,  Glaser  &  Strauss,  1967;  Miles  &  Huberman,  1994).  Using  a 
grounded  theory  approach,  data  will  be  reviewed  for  common  themes  and  specific  narratives  or  quotes 
that  can  be  used  to  illuminate  the  study  questions.  Data  reduction  will  occur  through  content  analysis 
with  corresponding  inter-rater  reliability  calculations  for  the  purpose  of  integrating  qualitative  and 
quantitative  findings.  Data  interpretation  will  be  negotiated  by  VU  and  FHP&R  personnel  to  ensure 
accuracy  in  findings,  with  results  used  to  inform  the  intervention  development  process. 

Aim  l.c.  Secondary  analysis  of  PDHRA  data  to  identify  variability  attributed  to  the  provider.  Under  their 
current  contract,  VU  is  applying  statistical  predictive  modeling,  a  commonly  used  public  health  approach 
to  identifying  individuals  at-risk.  The  goal  is  to  identify  potential  criteria,  or  thresholds,  for  risk  relevant 
to  overall  index  scores  and  specific  subscale  scores  derived  from  the  SM  self-report.  Predictive  modeling 
seeks  to  establish  relationships  between  sets  of  variables  in  order  to  predict  future  outcomes.  It  then 
forecasts  future  events  based  on  the  identified  relationships  (Cousins,  2002).  VU  is  in  the  process  of 
conducting  logistic  regression  models  (Roblin,  1999;  Schatz,  2003)  with  the  current  secondary  analysis 
dataset  representing  595,191  SMs  from  all  Service  Branches  and  Components.  While  the  dataset  is  rich 
in  terms  of  SM  characteristics  (e.g.,  combat  exposure,  number  of  deployments,  health  care  utilization), 
the  way  the  data  are  provided  at  present  allows  no  possibility  of  identifying  potential  patterns  based  on 
provider  characteristics.  Therefore,  we  propose  to  develop  additional  data  readily  available  at  the 
Service  level  to  be  incorporated  in  ongoing  secondary  analysis.  Specifically,  we  will  request  provider 
identification  information,  which  is  available  as  a  digital  variable  in  each  of  the  Service  medical  records 
systems  (i.e.,  AHLTA  for  the  Army,  NEHC  for  the  Navy).  VU  and  FHP&R  will  utilize  existing  relationships 
with  AMSA  and  the  Expert  Panel  to  identify  and  follow  appropriate  procedures  to  request  the  data  from 
information  technology  officers  at  each  Service.  As  per  IRB  protocols,  the  only  data  requested  for  the 
purposes  of  matching  the  PDHRA  with  the  health  care  utilization  records  will  be  the  unique  identifier 
(typically  the  social  security  number)  for  the  individual  SM  record  and  the  provider  identification 
variable.  A  dataset  with  these  two  variables  will  be  provided  directly  to  FHP&R.  Prior  to  providing  any 
data  to  VU,  all  records  will  be  de-identified  such  that  no  SMs  can  be  individually  identified.  Existing 
procedures  are  already  in  place  to  support  this  de-identification  procedure.  The  data  will  be  analyzed  in 
multivariate  logistic  regression  models.  Standard  criteria  for  evaluating  the  quality  of  health  risk 
appraisal  systems  include  sensitivity  and  specificity  and  goodness  of  fit. 

Sensitivity  and  Specificity.  The  sensitivity  of  a  model  is  its  ability  to  identify  those  at  risk,  while 
specificity  refers  to  a  model's  ability  to  identify  those  not  at  risk  of  the  outcome.  In  general,  sensitivity 
and  specificity  are  measures  that  assess  the  validity  of  diagnostic  and  screening  tests  (see  figure  below). 
Practically  speaking,  a  highly  sensitive  assessment  means  is  one  in  which  a  large  percentage  of  the 
population  are  classified  correctly  as  having  the  disorder;  a  highly  specific  assessment  is  one  in  which 
individuals  without  the  disorder  in  question  are  not  incorrectly  identified  as  having  the  disorder.  Ideal 
screening  assessments  are  maximally  sensitive  and  specific,  where  100%  of  individuals  at-risk  would  be 
detected  and  risk  would  be  ruled  out  in  100%  of  those  who  are  truly  not  at  risk.  This  framework  is  useful 
for  evaluating  decision  rules  or  cut  points  for  a  measure  because  it  accurately  reflects  how  an  increase 
on  any  one  of  these  indices  tends  to  co-occur  with  a  decrease  on  another.  In  fact,  during  development, 
researchers  often  use  conditional  probability  analysis  values  to  plot  sensitivity  against  specificity  using  a 
range  of  cut  scores  to  determine  the  ideal  decision  rules  for  classifying  individuals  into  at-risk,  or  not-at 
risk  groups.  Both  sensitivity  and  specificity  are  intimately  related  to  both  positive  and  negative 
predictive  values. 

Calculation  of  Sensitivity  and  Specificity 
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The  positive  predictive  value  (PV+)  is  the  percent  of  positive  tests  that  are  truly  positive.  The  negative 
predictive  value  (PV-)  is  the  percent  of  negative  tests  that  are  truly  negative.  Like  sensitivity  and 
specificity,  PV+  and  PV-  also  show  how  well  the  test  is  classifying  individuals  into  disease  and  non¬ 
disease  groups,  but  the  denominator  for  PV+  is  the  total  number  of  persons  who  test  positive  (a  +  b), 
while  that  for  PV-  is  the  total  number  who  test  negative  (c  +  d).  (Figure  2)  A  test  with  a  high  PV+  value 
means  that  there  is  only  a  small  percent  of  false-positives  within  all  the  individuals  with  positive  test 
results.  A  test  with  a  high  PV-  value  means  that  there  is  only  a  small  percent  of  false-negatives  within  all 
the  individuals  with  negative  test  results. 

Goodness  of  Fit.  Indices  of  the  "Goodness  of  Fit"  are  most  commonly  used  to  describe  the 
quality  of  predictive  models  of  risk  or  how  well  these  regression-based  models  'fit'  the  actual  data 
observed.  Predictive  models  derived  from  linear  regression  typically  use  as  the  primary  'goodness  of 
fit"  index,  whereas  the  goodness  of  fit  of  models  based  on  logistic  regression  are  typically  measured 
using  receiver  operating  characteristic  (ROC)  curves  (Flosmer,  2000).  R-squared  (R^)  indicates  the 
percentage  of  total  variation  among  individual  observations  that  can  be  explained  by  the  model,  either 
explained  as  a  percentage  or  a  number  between  0  and  1;  0  explains  none  of  the  variance,  1  explains  all 
the  variance  (Flu,  2004).  Thus,  the  closer  a  model's  R-squared  value  is  to  1,  the  better  the  predictive 
model  explains  the  data.  For  logistic  regression  models,  the  relationship  between  sensitivity  and 
specificity  is  usually  explored  using  a  ROC  curve.  To  construct  the  ROC  curve,  the  x-axis  is  1  minus  the 
specificity  (false  positive)  and  the  y-axis  is  the  sensitivity  (true  positive  rate;  Crichton,  2002).  The  area 
under  the  curve  (AUC  or  c-statistic)  can  summarize  the  capacity  of  a  model  for  discriminating  those  who 
experience  the  event  of  interest  (for  example,  risk  of  admission)  versus  those  who  do  not,  and  can 
therefore  be  used  to  compare  models  (Liu,  2003). 

PDFIRA  Model  to  be  Tested.  As  a  multi-gate  health  risk  appraisal  process,  the  PDFIRA  can  be 
tested  with  layered  outcomes  based  on  the  initial  SM  self-report  of  problems.  For  the  PDFIRA,  the 
proximal  evaluation  criteria,  or  outcome  can  be  defined  as  whether  or  not  a  referral  was  made  as  part  of 
the  process.  A  distal  evaluation  outcome  is  derived  from  review  of  subsequent  SM  health  care  utilization 
over  the  eight  week  period  immediately  following  the  PDFIRA.  Utilization  data  include  completion  of 
appointment,  date  of  appointment,  and  primary  and  secondary  ICD-9  diagnostic  codes. 

From  the  SM  self-report  of  problems,  there  are  five  specific  subscales  within  the  PDFIRA  relevant  to 
behavioral  health  and  concussion  concerns: 

•  mTBI  symptoms 

•  PTSD  symptoms 

•  Depressive  symptoms 

•  Alcohol  use  problems 

•  Relationship  conflicts 
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From  all  the  data  available,  there  are  three  primary  areas  to  explore  in  assessing  quality  of  the  health 
risk  appraisal  process: 

•  Concordance  between  SM  self-reported  problems,  provider-reported  concerns,  and  referral 
characteristics 

•  Factors  impacting  SM  acceptance  of  referral 

•  Concordance  between  PDFIRA  and  subsequent  health  care  utilization  (SM  self-reported 
problems,  provider  concerns,  referrals) 

Aim  l.d.  Study  of  Risk  Factors  Associated  with  the  Development  of  PTSD. 

Sample  and  Data.  The  eligible  study  population  will  be  Service  Members  (SMs)  with  at  least  one 
deployment  to  OEF/OIF  of  greater  than  30  days  duration.  The  DoD  and  VA  will  identify  eligible  SMs  from 
their  medical  records.  Both  agencies  will  de-identify  data  and  assign  a  unique  study  identification 
before  releasing  the  data  to  analysts  for  analysis. 

Study  Design.  An  observational,  retrospective  case-control  design  will  be  utilized. 

"Cases"  will  be  defined  as  eligible  SMs  who  at  some  time  after  first  deployment  meet  the  current  criteria 
for  confirmed  PTSD  (at  least  one  in-patient  encounter  or  two  outpatient  encounters  with  a  diagnostic 
code  indicating  PTSD  (ICD=309.81)).  Medical  encounter  data  for  implementing  these  criteria  will  include 
direct  care  (SIDR,  SADR),  purchased  care  (TED-1,  TED-1)  and  VA  data. 

"Controls"  will  be  selected  randomly  at  the  rate  of  4  eligible  per  case.  Each  control  will  have  been 
deployed  for  at  least  six  months  prior  to  the  PTSD  diagnostic  confirmation  for  the  case  and  not  meet  the 
criteria  for  confirmed  PTSD  or  other  mental  health  disorder  at  the  time  of  case  determination.  Controls 
will  be  matched  with  cases  on  the  following  variables:  gender.  Service  component,  military  occupational 
specialty  -  factors  that  could  be  associated  with  PTSD  but  are  not  being  examined  in  this  study. 

Variables  of  interest  (independent  variables)  in  this  study  will  include  the  number  of  deployments,  the 
total  time  deployed,  average  time  for  each  single  deployment,  dwell  time  between  deployments, 
previous  healthcare  utilization  patterns,  pre-existing  medical/mental  health  conditions,  reports  and 
referrals  on  health  assessments  including  PDFIA,  PDFIRA,  PFIA,  separation  exams,  VFIA  entrance  exams, 
clinical  health  screening  results,  frequency/intensity  of  combat  exposure,  marital/family  status,  and  age, 
rank,  time  in  service  and  their  association  with  the  development  of  PTSD. 

Statistical  Analysis.  Analyses  will  be  conducted  using  Stata  (Version,  10,  College  Station  Texas). 

A  multivariate  analysis  of  variance  will  be  performed  with  logistic  regression  to  identify  variables 
significantly  associated  with  the  development  of  PTSD. 

Aim  2:  Evaluate  the  effectiveness  of  a  targeted  training  and  feedback  program  on  primary  care 
provider's  interview  and  clinical  communication  patterns  related  to  SM  behavioral  health  condition 
identification  and  referrals. 

Based  on  the  findings  from  Aim  1,  a  feedback  and  training  intervention  for  providers  will  be  tested  with 
a  group  of  39  primary  care  providers  who  conduct  deployment-related  assessments  in  four  to  six  sites. 
Providers  will  be  randomly  assigned  to  one  of  two  interventions  (initial  training  and  ongoing  feedback  or 
initial  training  only)  or  to  usual  training  procedures,  with  usual  training  serving  as  the  control  arm  of  the 
study. 
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Training  and  Feedback  Intervention 

While  the  specific  content,  format,  and  implementation  of  the  intervention  will  be  developed  as 
described  in  Aim  1  above,  we  will  follow  a  multi-modal  approach  to  training  including  an  initial 
workshop,  a  written  guide,  and  ongoing  peer  learning  utilizing  case  review.  This  comprehensive  training 
package  is  proposed  in  response  to  the  ample  evidence  that  one-time  trainings  or  provision  of  manuals 
alone  is  insufficient  to  change  practice  behavior  (e.g.,  Davis  et  al.,  1995;  Davis  et  al.,  1999;  Oxmen  et  al., 
1995). 

The  initial  training  will  consist  of  a  one-day  8-hour  workshop  to  include  the  following  content  areas: 

•  Review  of  findings  from  VU-FHP&R  collaborative  project  as  relevant  to  the  PDHRA  clinical 
interview 

•  Role  of  the  provider  in  the  deployment  health  assessment  process 

•  Individual  barriers  to  self-reporting 

•  Interview  and  communication  techniques  to  elicit  candid  reporting  of  behavioral  health 
problems 

•  Identification  of  behavioral  health  problems  that  warrant  referral 

•  Motivational  techniques  to  increase  referral  acceptance 

The  initial  workshop  will  be  interactive,  including  didactics  and  group  discussion.  Cases  will  be  presented 
using  multiple  media  (audio  and  visual).  A  written  guide  will  be  provided  as  a  supplement  to  the  initial 
training,  including  information  and  scholarly  readings  relevant  to  all  of  the  content  outlined  above. 
Towards  the  close  of  the  workshop,  each  participant  will  be  asked  to  conduct  a  simulated  PDHRA 
interview  with  a  peer  for  formative  assessment  and  feedback. 

In  the  feedback  plus  initial  training  group  an  ongoing  peer  learning  will  follow  a  treatment  team  format 
and  will  likely  occur  on  a  weekly  or  bi-weekly  schedule,  depending  on  staffing  patterns,  PDHRA 
flowthrough,  etc.  Each  case  review  meeting  will  be  scheduled  for  30-45  minutes  and  led  by  the  PDHRA 
program  manager.  The  group  will  review  current  cases  for  group  discussion  to  provide  ongoing, 
experience-based  learning  relevant  to  the  outlined  content  above.  Each  participating  provider  will  be 
asked  to  share  3-5  audiotapes  throughout  the  study  period  to  be  used  in  the  case  review  meetings  to 
elicit  feedback  for  strengthening  communication  and  interviewing  skills.  Feedback  will  continue  over  the 
study  period  of  4  months. 

To  build  in  sustainability  from  the  beginning  and  increase  feasibility  of  large  scale  roll  out  after  the  study 
has  ended,  we  will  utilize  a  train-the-trainer  model  in  all  aspects  of  the  intervention.  Each  site  will  be 
asked  to  identify  one  or  two  providers  who  can  serve  as  intervention  facilitators.  These  providers  will  co¬ 
lead  the  initial  workshop  and  provide  on-site  leadership  for  the  ongoing  case  review.  The  project  team 
will  be  on-site  co-leading  the  initial  workshop  and  be  available  by  teleconference  for  the  case  review 
meetings  in  a  coaching  capacity.  Intervention  facilitators  will  be  considered  local  experts,  and  training 
materials  will  be  provided  to  them  prior  to  the  on-site  training  visit.  They  will  also  participate  in  training 
review  meeting  prior  to  the  on-site  training. 

Random  Assignment  to  Groups 

To  test  the  effectiveness  of  the  intervention,  we  propose  random  assignment  to  three  groups: 

•  Initial  Training  and  Ongoing  Feedback 

•  Initial  Training  only 

•  Training  as  usual  (control  group) 
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Implementation  Measures 

Implementation  fidelity  will  be  measured  through  attendance  records  for  the  initial  workshop  and  the 
ongoing  case  review,  where  applicable.  Implementation  quality  will  be  measured  through  brief  pre-  and 
post-training  evaluations  for  the  initial  workshop.  At  the  end  of  the  study,  we  will  administer  a  third 
evaluation  survey  to  be  completed  by  providers.  The  evaluations  will  measure  the  standard  issues 
related  to  training  (adequacy  of  setting  and  presentation;  met  learning  objectives;  etc.).  In  addition,  they 
will  also  attempt  to  gauge  the  perceived  usefulness  of  the  training  and  knowledge  gained.  Additional 
data  for  implementation  quality  will  be  obtained  by  taping  the  simulated  interviews  during  the  initial 
training  workshop  and  content  coding  similar  to  the  outcome  measures  described  below. 

Outcome  Measures 

A  multi-method  approach  to  outcome  measurement  is  proposed  to  increase  the  depth  of  findings, 
including: 

•  Pre-  and  post-training  content  analysis  of  audiotapes 

•  Brief  SM  satisfaction  survey 

•  SM  PDHRA  (DD  Form  2900) 

•  SM  subsequent  health  care  utilization  over  eight  week  period 

Audiotape  Content  Analysis.  Each  provider  will  be  asked  to  audiotape  one  hour  of  interviews 
prior  to  the  random  assignment  and  initial  training  workshop  and  one  hour  each  month  during  the  4- 
month  study  period,  for  a  total  of  five  measurement  intervals.  We  anticipate  an  average  of  five 
individual  interviews  per  one  hour.  Taping  dates  and  times  will  be  randomly  selected.  Using  staff 
schedules  provided  by  the  PDHRA  program  manager,  we  will  ask  each  provider  to  tape  at  a  certain  time, 
with  a  backup  time  to  be  used  as  needed.  Prior  to  the  week  of  scheduled  taping,  VU  will  ship  a  digital 
audio  recorder  to  the  provider  with  return  shipping  provided.  Following  the  taping,  the  provider  will  ship 
the  entire  audio  recorder  directly  to  FHP&R,  where  they  will  be  de-identified  prior  to  providing  the 
recordings  to  VU  for  analysis.  The  audiotapes  will  be  content  coded  using  the  RIAS  (Roter  &  Larsen, 

2002)  as  described  above  for  specific  utterances  related  to  socio-emotional  and  task-oriented  exchanges 
relevant  to  provider  communication  and  interview  style.  The  criteria  established  in  Aim  1  in 
collaboration  with  the  Expert  Panel  will  be  used  to  evaluate  findings. 

Brief  SM  Satisfaction  Survey.  A  five  to  seven  question  written  satisfaction  survey  will  be  handed 
to  each  SM  by  the  health  care  provider  as  they  exit  from  the  PDHRA  clinical  interview.  The  survey  will  be 
anonymous.  The  provider  name,  date  and  time  of  interview  will  be  preprinted  on  the  survey.  The  survey 
will  assess  SM  satisfaction  with  the  overall  PDHRA  process  and  comfort  answering  honestly  on  the  self- 
report  and  in  the  provider  interview.  These  forms  will  be  collected  by  the  PDHRA  program  manager  and 
shipped  to  VU  on  a  bi-weekly  basis  (or  more  frequently  in  cases  with  significant  throughput).  Data  will 
be  double-entered  at  VU  to  ensure  data  quality. 

Electronic  records  for  SM  PDHRA  and  Health  Service  Utilization.  We  anticipate  each  provider  will 
interview  an  average  range  from  100  to  500  SMs  during  a  typical  four-month  time  period,  depending  on 
PDHRA  throughput  and  deployment  patterns.  We  will  request  the  electronic  records  of  PDHRAs  (DD 
Form  2900)  and  subsequent  eight  weeks  of  health  service  utilization  records  for  each  SM  interviewed  by 
participating  providers  during  the  study  period.  We  will  also  request  clinician  and  MTF  identifying 
variables  directly  from  an  AHLTA  contact.  All  data  requests  will  be  conducted  using  establishing 
procedures.  The  electronic  records  provide  important  variables  for  use  in  determining  appropriateness 
of  referral  from  the  perspective  of  the  SM  (self-report  of  problems,  acceptance  of  referral),  the  provider 
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(noted  concerns,  referral  made),  and  actual  health  care  utilization  (appointment  completion,  ICD-9 
diagnostic  codes). 

Analysis  Plan 

Aim  2  hypothesizes  that  PCPs  given  enhanced  training  or  training  +  feedback  will  conduct  more  probing 
health  screens,  show  improved  sensitivity  to  detecting  problems,  and  more  accuracy  compared  with 
those  given  typical  training  for  the  screening  task.  The  analysis  plan  has  two  general  cases  for  all  the 
study's  outcomes.  For  PDHRA  outcomes,  PCPs  will  screen  SMs  on  an  average  of  12  days  during  the 
study's  4  months  after  training.  On  each  occasion,  the  PCP  evaluates  numerous  SMs  (e.g.  80/day), 
which  will  make  daily  estimates  of  PCP  screening  behavior  quite  reliable.  Audio  tape  analysis,  being 
much  more  expensive,  will  be  done  on  5  occasions  with  fewer  SM  interviews.  Both  sets  of  outcomes 
will  be  treated  as  longitudinal  (repeated  measure)  hypotheses. 

The  experimental  design  compares  the  3  treatment  groups  in  a  "slope  as  outcome"  design.  Since  the 
effects  of  training  or  feedback  would  take  time  to  occur,  we  hypothesize  cumulative  results  as 
exemplified  in  the  figure  below.  Having  multiple  repeated  measures  will  also  make  it  possible  to  see  if 
change  is  constant,  or  if,  for  example,  the  effects  of  training  are  lost  in  later  months.  At  this  point  our 
working  assumption  is  that  change  will  be  linear.  Before  analysis  we  will  examine  smoothed  individual 
timelines  of  PCPs  on  key  outcomes  to  see  what  the  shape  of  change  actually  may  be  (Singer  &  Willett, 
2003).  For  example,  with  training  there  may  be  an  immediate  dramatic  improvement  followed  by 
decline  as  old  habits  and  workarounds  recur.  The  effect  of  feedback  may  be  gradual,  but  superimposed 
on  a  nonlinear  training  curve.  When  clear  patterns  can  be  seen,  we  can  use  piecewise  longitudinal 
models  (Lambert,  Wahler,  Andrade,  &  Bickman,  2001),  and  if  the  meaning  of  the  nonlinearity  is  obscure, 
generic  splines  may  be  used  to  fit  arbitrary  growth  curves  (Harrell,  2001). 
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Analytic  model.  A  longitudinal  repeated 
measure  slope-as-outcome  model  will  be  used,  as 
recommended  by  recent  authors  (Hedeker  & 

Gibbons,  2006)  (Fitzmaurice,  Laird,  &  Ware,  2004). 

This  approach  is  called  by  various  names  such  as 
random  regression,  multilevel  modeling,  hierarchical 
linear  modeling  (HLM),  multilevel  modeling,  or  mixed  modeling.  This  approach  offers  important 
advantages  over  older  analysis  of  variance  (ANOVA)  models  (Nich  &  Carroll,  1997),  such  as  better 
handling  of  missing  values  and  unequal  time  intervals  between  waves  and  subjects.  Longitudinal  models 
are  necessary  for  because  the  repeated  measurements  of  a  given  PCP  will  violate  the  assumption  of 
independent  errors  that  ANOVA  requires.  This  lack  of  independence  should  be  modeled,  not  ignored. 
Other  advantages  include:  a)  Repeated  measurements  adding  to  statistical  power,  b)  describing  the 
shape  of  change  over  time,  and  c)  avoiding  the  psychometric  problems  with  pre-post  change  scores 
(Lambert,  Doucette,  &  Bickman,  2001).  By  the  late  1990's  dependable  software  was  widely  available, 
e.g.  from  SAS  (Littell,  Milliken,  Stroup,  et  al.,  1996),  Bryk  &  Raudenbush  (1992),  and  others  (Pinheiro  & 
Bates,  2000). 

For  simplicity,  power  analysis  was  done  on  a  linear  slope  as  outcome  model  using  a  longitudinal  power 
analysis  (Diggle,  Fleagerty,  Liang,  &  Zeger,  2002).  A  hypothetical  outcome  with  a  starting  mean  of  50  (SD 
=  10)  was  used  to  represent  any  of  the  PDFIRA  variables.  Without  empirical  data  on  effect  sizes  in  this 
situation,  we  estimate  the  minimum  detectable  effect  size,  as  recommended  by  Kraemer  et  al. 

(Kraemer,  Mintz,  Noda,  Tinklenberg,  &  Yesavage,  2006).  We  estimate  how  small  a  change  over  time 
could  be  detected  with  traditional  standards  (Cohen,  1988)  of  5%  alpha  level  (two-tailed)  and  80% 
power.  The  power  analysis  assumed  13  repeated  measurements,  3  groups  of  13  PCPs:  Control,  Training, 
and  Training  +  FB.  Not  knowing  the  temporal  stability  we  will  find  in  these  observations,  we  assumed  an 
average  cross-wave  correlation  of  r  =  0.  66  based  on  our  other  research  with  repeated  measurements. 
According  to  the  Diggle  model,  an  endpoint  difference  as  small  as  Cohen's  d  =  (Mi  -  M2)  /  SDpooied  =  0-50 
SDs  between  treated  and  untreated  PCPs  could  be  detected  with  adequate  power.  According  to  (Cohen, 
1992),  this  is  a  medium  sized  effect. 

The  same  power  analysis  was  run  for  the  5-wave  analysis  of  audio 
tape  data.  Flaving  fewer  repeated  measurements  lowered  power  and 
made  the  minimum  detectable  effect  size  larger,  in  this  case  d  =  0.76 
at  the  endpoint.  According  to  Cohen,  0.80  SDs  is  a  "large"  effect. 

Given  the  expense  of  any  change  in  administration  of  the  PDFIRA  we 
believe  that  medium  and  large  effects  are  reasonable  minimum  effect 
sizes.  Moreover,  increasing  sample  size  as  an  approach  to  increasing 
power  would  make  the  study  much  more  expensive. 

B.  4  Collaborating  Institution 

The  Center  for  Evaluation  and  Program  Improvement  (CEPI)  at 

Vanderbilt  University  (VU)  has  a  long  and  extensive  history  of  successfully  completing  large  scale  and 
complex  mental  health  services  research.  Almost  all  of  this  work  has  been  supported  by  NIMFI  or  some 
branch  of  the  military. 

Leader  in  Mental  Flealth  Services  Research 

The  Ft.  Bragg  demonstration  project  was  and  still  is  the  largest  evaluation  of  a  system  of  care,  lasting 
over  five  years  with  80  million  dollars  in  funding.  A  quasi-experimental  study  of  services  for  military 


Longitudinal  power  analyses 
had  a  minimum  detectable 
endpoint  effect  size  of  0.50 
SDs  (PHDRA)  and  0.76  SDs 
(taped  outcomes).  The 
inability  to  detect  small 
changes  is  not  a  problem 
because  training  or  feedback 
that  produced  only  small 
improvements  should  be 
counted  as  a  failure. 


Analysis  of  each  outcome  will  be  a  "slope  as 
outcome"  design  in  which  the  effects  of 
treatment  develop  over  time  between  3  groups 
of  PCPs  randomly  assigned  to  Feedback, 
Enhanced  Training,  or  Traditional  Orientation. 
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dependents,  the  project  compared  an  enhanced  system  of  care  (Ft.  Bragg)  to  services  as  usual  (Ft. 
Campbell  and  Ft.  Stewart).  We  found  that  youth  in  the  system  of  care  had  no  better  outcomes  and  cost 
more  than  treatment  as  usual  (Bickman  et  al.,  1995).  This  study  and  the  follow-up  research  have  been 
nationally  recognized  for  their  quality  with  awards  from  several  organizations.  Major  figures  in 
evaluation  such  as  Thomas  Cook  characterized  them  as  "among  the  10  or  20  best  evaluation  studies 
ever  done  in  any  field  by  anyone".  Carol  Weiss  called  it  "one  of  the  landmark  studies  of  the  decade" 
(Fitzpatrick,  2002,  p.  65).  We  received  a  competitive  renewal  from  NIMFI  and  additional  funds  from  DoD 
to  extend  the  data  collection  two  more  years  and  conduct  a  large  number  of  secondary  analyses.  One 
study  of  PTSD  found  that  clients  with  a  diagnosis  of  PTSD  had  greater  severity  of  psychopathology, 
poorer  functioning,  and  higher  treatment  intensity  compared  to  others  with  Axis  II  mental  health 
disorders  (Kelley  &  Bickman,  1998).  We  followed  up  this  study  with  a  partial  replication  of  Ft.  Bragg  in 
Stark  County,  Ohio  using  a  randomized  design  (Bickman,  Summerfelt,  Firth,  &  Douglas  [a.k.a.  Kelley], 
1997;  Bickman  et  al.,  1999).  The  results  of  the  Stark  County  study  were  the  same  as  in  Ft.  Bragg.  A  third 
study  evaluated  a  Wraparound  Demonstration  for  DoD  with  the  same  results  (Bickman  et  al.,  2003).  In  a 
funded  follow-up  study  to  the  Stark  County  project,  we  conducted  qualitative  interviews  with  caregivers 
and  clinicians  to  inform  quantitative  findings  (Noser  &  Kelley,  1998).  We  have  conducted  mixed  methods 
evaluations  of  settings  ranging  from  hospital-based  psychiatry  clinics  (Kelley,  Van  Florn,  &  DeMaso, 

2001)  to  post-combat  environments  (Kelley,  McDonald,  &  Mollica,  2005). 

Our  studies  revealed  that  system  level  interventions  could  affect  system  level  variables  like  cost  and 
access  but  did  not  affect  clinical  outcomes  unless  improvements  were  made  in  clinical  services.  The 
disappointing  results  of  these  evaluations  provided  the  impetus  for  our  focus  on  clinician  behavior  as 
the  critical  mediator  client  of  outcomes.  Therefore,  we  developed  a  conceptual  approach  to  clinician 
change  that  was  grounded  in  empirical  literature.  Based  on  that  theory  we  then  developed  and  tested  a 
feasible  method  to  promote  clinician  change. 

Interventions  to  Change  Provider  Behavior 

Over  the  past  several  years,  we  have  developed  a  theory  and  measurement  system  to  support  a 
conceptual  model  of  provider  behavior  change.  We  have  tested  or  are  testing  the  effectiveness  of 
feedback  on  provider  behavior  in  several  settings.  In  a  NIMFI  funded  study  (5R01MFI62951),  we  applied 
research  on  feedback  in  an  attempt  to  change  the  practice  behavior  of  office  based  physicians.  We  also 
have  been  successful  in  implementing  a  self-efficacy  enhancing  intervention,  another  key  element  of  our 
theory  (Bickman  et  al.,  1998).  A  second  study,  funded  by  the  U.S.  Department  of  Education,  investigated 
the  influence  of  feedback  on  fidelity  of  implementation  on  teacher  behavior  as  part  of  a  larger  multi-site 
study  of  school-based  character  education.  A  third  NIMFI  funded  study  (RO1-MFI068589)  uses  a 
longitudinal  factorial  randomized  experimental  design  to  determine  the  effectiveness  of  feedback  on 
mental  health  provider  behavior  and  subsequent  client  outcomes.  A  fourth  study,  funded  by  the  U.S. 
Department  of  Education,  focuses  on  testing  the  effectiveness  of  feedback  from  teachers  and  coaching 
on  that  feedback  on  school  principal  behavior. 

These  studies  sensitized  us  to  the  importance  of  understanding  the  context  in  which  the  intervention 
was  to  take  place.  Neither  settings  nor  providers  are  standard  units  that  can  be  interchanged  (Peterson 
&  Bickman,  1989).  We  have  also  developed  a  comprehensive  approach  to  training  given  the  evidence 
that  a  training  manual  or  a  workshop  by  itself  will  result  not  result  in  clinician  behavior  change.  Based  on 
a  literature  review  and  experience  gained  from  state-  and  foundation-funded  professional  development 
initiatives  (Flawley  &  Kelley,  2005;  Kelley,  Winson,  &  Affrunti,  2005)  our  training  package  includes 
multimodal  presentation  of  information  (live,  online,  and  written),  ongoing  coaching  and  support,  and 
emphasis  on  clinically  useful  material,  all  tailored  to  the  local  setting. 
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Current  Collaboration  with  FHP&R 

CEPI  is  currently  contracted  with  DoD  (contract  #  W81XWH-07-P-1026;  9/07-9/09)  to  conduct  a  two- 
year  evaluation  of  the  PDHA/PDHRA  process  encompassing  all  Branches  and  Components  of  the 
military.  The  overall  goals  of  this  project  are  to  evaluate  if  the  health  risk  appraisal  process  increases 
access  to  appropriate  care  for  Service  Members  in  need,  with  a  focus  on  under-reporting  of  symptoms 
on  the  SM  self-report  and  providers'  ability  to  elicit  reporting  of  symptoms  and  acceptance  of  referrals 
during  the  clinical  interview.  Thus  we  seek  to  identify  SM-  and  clinician-related  factors  that  influence 
PDHRA  outcomes  and  understand  the  nature  of  the  influence.  To  this  end,  we  use  a  mixed  methods 
approach  including  surveys  of  SMs,  interviews  with  Unit  leaders,  PDHRA  program  managers,  and  other 
key  individual  involved  in  the  PDHRA  process,  a  secondary  analysis  of  PDHA,  PDHRA,  and  health  service 
utilization  data,  and  an  analysis  of  audiotapes  of  health  care  provider  interviews  completed  over  the 
telephone.  The  secondary  analysis  and  analysis  of  telephone  screening  interviews  are  of  particular 
interest  for  the  proposed  study,  as  described  previously. 

C.  Military  Relevance  Statement 

The  proposed  study  will  directly  respond  to  the  psychological  health  care  needs  and  quality  of 
life  of  military  personnel  returning  from  combat/operational  deployments.  The  identification 
and  referral  of  potential  mental  health  conditions  and  concerns  has  been  an  issue  of  much 
debate  within  the  DoD  and  across  the  government. 

The  best  method  to  ensure  accurate  and  useful  assessment  of  conditions  is  the  first  critical 
piece.  This  study  will  engage  a  comprehensive,  multi-method  approach  that  has  not  been  used 
in  the  past  to  examine  this  population.  First,  by  using  a  cohort  of  individuals  who  have  been 
diagnosed  with  PTSD  in  both  the  DoD  and  VA,  we  can  look  back  in  their  history  to  determine 
what  contributory  factors  were  associated  with  that  diagnosis.  Second,  by  examining  the  actual 
referral  process  for  engaging  individuals  in  mental  health  treatment,  we  can  make  sure  that 
those  who  really  need  evaluation  and  treatment  actually  are  persuaded  to  seek  the  appropriate 
care.  Thus,  we  will  create  a  continuum  of  more  accurate  assessment  and  more  compliance 
with  referrals  resulting  from  that  assessment. 

Once  we  know  the  optimal  process  for  referral  and  communication  with  patients  that 
encourages  them  to  overcome  perceived  stigma  and  seek  care,  we  can  train  our  providers  to 
provide  better  clinical  health  risk  communication  that  will  motivate  those  who  are  identified  as 
in  need  of  treatment  to  actually  follow-though  with  referrals.  Early,  more  accurate 
identification  and  follow-through  on  treatment  promises  to  reduce  morbidity  and  improve 
treatment  outcomes,  returning  more  of  our  service  members  to  duty  and  more  of  our  veterans 
to  fulfilling,  high  quality  lives. 

D.  Impact  Statement 

If  successful,  findings  from  the  proposed  study  will  help  assist  in  making  more  accurate  referrals 
and  more  effective  referrals  in  assessment  procedures  across  the  lifecycle  of  care  for  our 
service  members  and  veterans.  PTSD  and  other  mental  health  conditions  and  concerns  are 
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considered  by  leadership  and  medical  providers  as  being  significant  negative  health  effects  of 
the  current  conflict.  The  military  and  the  VA  have  been  pushing  aggressively  to  identify  and 
treat  these  conditions,  but  little  in  the  way  of  solid  science  has  been  offered  to  assist  in  that 
process,  in  part  due  to  time  constraints  and  to  incomplete  data  sets.  This  study  takes  a  major 
step  toward  closing  those  gaps.  It  will  impact  the  way  that  PTSD  and  other  mental  health 
conditions  are  identified  in  an  assessment  process  and  how  primary  care  providers  can  best 
communicate  with  those  with  concerns  to  ensure  they  get  the  treatment  they  need. 

If  the  study  is  not  conducted,  the  health  assessment  and  referral  process  will  proceed  without 
the  benefit  of  science  and  fact-based  decision  making,  rendering  them  less  effective  and  less 
useful  to  our  service  members  who  struggle  with  psychological  distress  and  mental  health 
disorders.  Military  psychological  readiness  will  also  suffer  as  individuals  who  are  referred  for 
and  in  need  of  treatment  may  choose  not  to  access  that  treatment  as  a  result  of  lack  of 
effective  communication  on  the  part  of  our  providers. 

E.  Transition  Plan 

The  findings  from  this  study  will  be  integrated  into  the  lifecycle  health  assessment  process 
including  the  PDHA,  PDHRA  and  PHA  as  well  as  clinical  screening  processes  to  better  enable  our 
healthcare  providers  to  care  for  our  service  members  and  veterans  with  negative  health  effects 
of  combat  operations.  In  turn,  as  better  identification  of  patients  reduces  false  positives  and 
the  true  positives  are  motivated  to  seek  care  early,  more  military  personnel  will  be  successfully 
treated  and  returned  to  functional  status  and  back  to  duty. 
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VI.  ACRONYMS  AND  SYMBOL  DEFINITIONS 


AHLTA  Armed  Forces  Health  Longitudinal  Technology  Application 

ANOVA  analysis  of  variance 

AUC  area  under  the  curve 

CEPI  Center  for  Evaluation  and  Program  Improvement 

DC  District  of  Columbia 


DoD  Department  of  Defense 

FB  feedback 

FHP&R  Force  Health  Protection  and  Readiness 

HLM  hierarchical  linear  modeling 

lAA  Institutional  Authorization  Agreement 

ICD-9  International  Statistical  Classification  of  Diseases  and  Related  Health 

Problems,  9*''  edition 

IRB  Institutional  Review  Board 

MRMC  Medical  Research  and  Materiel  Command 

mTBI  mild  traumatic  brain  injury 

MTF  Military  Treatment  Facility 

NEHC  Navy  Environmental  Health  Center 

NIMH  National  Institute  of  Mental  Health 

OEF  Operation  Enduring  Freedom 

OIF  Operation  Iraqi  Freedom 

PCP  Primary  Care  Provider 

PDHA  Post  Deployment  Health  Assessment 

PDHRA  Post  Deployment  Health  Reassessment 

PI  primary  investigator 

PTSD  Posttraumatic  Stress  Disorder 


PV+ 

PV- 

RIAS 

ROC 

SASE 

SD 

SES 

SM 

TMA 

USDHHS 

VU 

VII. 


positive  predictive  value 
negative  predictive  value 
Roter  Interaction  Analysis  System 
receiver  operating  characteristic 
self  addressed,  sealed  envelopes 
standard  deviation 
socioeconomic  status 
service  member 
TRICARE  Management  Activity 
U.S.  Department  of  Health  and  Human  Services 
Vanderbilt  University 
STATEMENT  OF  WORK 


A.  Vanderbilt  University  Statement  of  Work 
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Extramural  Partner  Site  (no  animal  or  human  use  at  this  site) 

Susan  Douglas  Kelley,  Ph.D.  (PI)  and  Leonard  Bickman,  Ph.D.  (Co-PI) 

Center  for  Evaluation  and  Program  Improvement,  Vanderbilt  University 

Study  Sites 

The  training  and  feedback  intervention  will  be  tested  in  four  to  six  study  sites.  The  number  of  sites  will 
depend  on  staffing  and  PDHRA  throughput.  Site  recruitment  will  be  limited  to  Army  Active  Duty.  Specific 
sites  will  be  determined  in  collaboration  with  the  Expert  Panel  in  the  first  two  months  of  the  project.  An 
estimated  39  providers  in  total  throughout  the  four  to  six  study  sites  will  be  recruited  to  participate  in 
the  study. 


Overall  Project  Timeline 

Proposed  two-year  project  period  from  01  AUGUST  2009  to  31  JULY  2011. 
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Task  1.  Timing  of  Approvals  and  IRB  (Year  1,  months  1-11 


If  required  for  this  grant,  VU  and  FHP&R  will  build  on  excellent  working  relationships  based  on  our 
current  research  project.  Appropriate  Army  IRBs  may  include  Regional  Medical  Center  (RMC)  and/or 
installation  IRBs  depending  on  the  study  sites  recruited. 

la.  Multi-project  Institutional  Authorization  Agreement  (lAA)  submitted  to  TMA  Exempt  Determination 
Official  to  establish  the  VU  IRB  as  the  IRB  of  record  (Year  1,  months  1-2) 

lb.  PHDRA  secondary  analysis  protocol  (Year  1,  months  1-4) 

•  Submitted  to  VU  IRB,  estimated  review  time  for  non-human  subjects  protocol  (Year  1,  month  1) 

•  Submitted  to  TMA  Exempt  Determination  Official,  estimated  review  time  (Year  1,  months  3-4) 
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lc.  PDHRA  focus  group  protocol  (Year  1,  months  1-4) 

•  Submitted  to  VU  IRB,  estimated  review  time  for  exempt  protocol  (Year  1,  months  1-2) 

•  Submitted  to  appropriate  Army  IRBs  and  estimated  review  time  (Year  1,  months  3-4) 

ld.  Training  and  feedback  intervention  study  protocol  (Year  1,  months  3-10) 

•  Submitted  to  VU  IRB,  estimated  review  time  for  expedited  protocol  (Year  1,  months  3-4) 

•  Submitted  to  appropriate  Army  IRBs,  estimated  review  time  (Year  1,  months  5-11).  Note  that 
final  training  materials  will  be  submitted  for  review  in  months  10-11. 

Task  2.  (Aiml)  PDHRA  focus  groups  (Year  1,  months  1-2,  5-9) 

2a.  Recruitment  of  four  to  six  study  sites  (Year  1,  months  1-2) 

2b.  Development  of  focus  group  protocols  (Year  1,  month  1) 

2c.  Administration  of  2-hour  focus  groups  conducted  at  each  study  site  (Year  1,  months  5-7) 

2d.  Professional  transcription  of  focus  group  audiotapes  ongoing  as  each  completed  (Year  1,  months  5- 

7) 

2e.  Qualitative  analysis  will  be  ongoing  as  each  focus  group  completed  with  aggregation  of  findings  after 
all  completed  (Year  1,  months  5-9) 

2f.  Production  of  preliminary  reports  and  briefings  (Year  1,  months  8-9). 

Task  3.  (Aim  1)  PDHRA  secondary  analysis  (Year  1,  months  4-9) 

3a.  Data  requests  to  appropriate  information  technology  officer  at  each  Service  for  provider  and  MTF 
identifiers  for  PDHRAs  completed  between  01/01/06  to  05/51/09  (Year  1,  month  4) 

3b.  Linking  file  created  by  TMA  to  provide  de-identified  dataset  to  VU  containing  non-identifying  SM 
identifier  and  provider/MTF  identifiers  (Year  1,  month  4) 

3c.  Data  management  and  analysis  (Year  1,  months  5-7).  Abbreviated  analytic  timeframe  estimated 
because  we  will  be  adding  this  dataset  to  existing  clean  datasets  with  much  of  the  analytic  programming 
developed. 

3d.  Production  of  preliminary  reports  and  briefings  (Year  1,  months  8-9). 

Task  4.  (Aim  2)  Training  and  feedback  intervention  effectiveness  study  (Year  1,  months  1-9;  Year  2, 
months  1-11) 

4a.  Recruitment  of  four  to  six  study  sites  (Year  1,  months  1-2) 

4b.  Development  of  training  materials  (Year  1,  months  1-9) 

4c.  Randomization  of  39  providers  across  four  to  six  study  sites  (Year  1,  month  12) 

4d.  Collection  of  pre-training  audiotapes  from  39  providers,  consisting  of  one  randomly  selected  hour  of 
PDFIRA  interviews  (Year  1,  month  12) 

^Training  and  feedback  intervention  (Year  2,  months  1-4) 

•  Initial  training  eight-hour  workshop  for  providers  in  the  two  intervention  conditions  (Year  2, 
month  1) 

•  Feedback  through  ongoing  peer  learning  in  treatment  team  format  conducted  at  relevant  study 
sites  for  30-45  minutes  on  a  weekly  or  bi-weekly  schedule  (TBD)  (Year  2,  months  2-4) 

4f.  Measurement  of  implementation  fidelity  and  quality  (Year  2,  months  1-4) 

•  Collection  of  initial  training  workshop  attendance  records,  administration  of  pre-  and  post¬ 
workshop  evaluations  completed  by  attending  providers,  and  audiotaping  of  simulated 
interviews  conducted  by  providers  during  initial  workshop  (Year  2,  month  1) 

•  Collection  of  attendance  records  at  ongoing  treatment  team  sessions  (Year  2,  months  2-4) 

•  Administration  of  post -training  evaluation  survey  to  participating  providers  (Year  2,  month  4) 

4g.  Measurement  of  intervention  outcomes  (Year  2,  months  1-4,  7) 
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•  Collection  of  audiotapes  from  39  providers,  consisting  of  one  randomly  selected  hour  of  PDHRA 
interviews,  one  each  month  of  the  study  period  (Year  2,  months  1-4) 

•  Administration  of  SM  satisfaction  survey  for  each  SM  participating  in  a  PDHRA  interview  with 
participating  providers  during  the  study  period  (Year  2,  months  1-4) 

•  Data  requests  to  Army  information  technology  officer  at  each  installation  for  provider  and  MTF 
identifiers  for  PDHRAs  completed  by  participating  providers  during  study  period  (Year  2,  month 
7) 

•  Data  request  to  TMA  for  (1)  de-identified  PDHRAs  completed  during  study  period  for 
participating  providers  during  study  period,  and  (2)  de-identified  health  care  utilization  records 
for  SMs  interviewed  by  participating  providers  for  eight  weeks  post-PDHRA.  Linking  file  will  be 
created  by  TMA  to  provide  de-identified  dataset  to  VU  containing  non-identifying  SM  identifier 
and  provider/MTF  identifiers  (Year  2,  month  7) 

4h.  Data  management  and  analysis  (Year  2,  months  5-11) 

Task  5.  Expert  Panel  meetings  (Year  1,  months  2,  9;  Year  2,  months  2, 11) 

The  development  of  the  intervention  will  guided  by  an  expert  panel  composed  of  SMs  in  leadership 
positions  related  to  the  PDHRA  process  from  all  Services. 

5a.  Four-hour  in-person  meeting  in  Washington,  DC  (Year  1,  month  1;  Year  2,  month  11) 

5b.  Two-hour  teleconference  calls  (Year  1,  month  9;  Year  2,  month  2) 

Task  6.  Project  planning  meetings  (Years  1  and  2,  all  months) 

6a.  Weekly  one-hour  teleconference  calls 

^  Three  one-day  intensive  project  meetings  to  be  held  at  FHP&R  in  Washington,  DC  (Year  1,  month  1, 
5,  10;  Year  2,  month  2,  6,  11) 

Task  7.  Preparation  of  final  reports  (Year  2,  months  11-12) 

Final  reports  and  briefings  will  be  prepared  according  to  any  guidelines  or  requirements  as  set  forth  by 
the  granting  agency. 


B.  Purdue  University  Statement  of  Work 

Year  1 

During  the  first  few  months  of  the  first  year  (09/09-11/09),  the  majority  of  the  work  will  be  conducted 
by  personnel  at  the  Roudebush  VA  Medical  Center  in  Indianapolis  and  at  the  Department  of  Defense  in 
Washington,  DC.  The  PI  (Mustillo)  will  establish  data  collection  procedures,  coordinate  data  collection 
between  those  two  sites,  troubleshoot,  and  assist  with  technical  consultation.  During  the  fall  of  2009, 
she  will  make  trips  to  Indianapolis  and  Washington  DC  as  necessary  to  consult  with  project  personnel 
regarding  data  and  data  collection.  In  12/09,  Mustillo  will  obtain  the  data  from  both  the  VA  and  the 
DoD  and  create  a  dataset  for  the  project.  During  the  spring  (01/10-05/10)  and  summer  (06/10)  Mustillo 
will  clean,  recode,  merge,  and  reshape  data  and  will  begin  data  analyses.  Mustillo  will  also  supervise  a 
research  assistant  to  help  with  these  tasks. 

Year  2 

In  Year  2,  Mustillo  will  continue  to  conduct  data  analyses  in  accordance  with  the  specific  aims  of  the 
project,  with  the  help  of  a  graduate  research  assistant  (08/10-12/10).  Additionally,  Mustillo  and  the 
research  assistant  will  write  a  report  of  the  finding  for  the  Department  of  Defense  as  well  as  manuscripts 
for  publication  in  professional  journals  (01/11-07/11).  Mustillo  will  travel  to  Washington  DC  to  consult 
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with  DoD  personnel,  present  key  findings,  and  receive  input  on  analyses  as  necessary.  Mustillo  also  will 
present  at  least  one  manuscript  at  a  research  conference. 
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IMPROVING  DEPLOYMENT  -  RELATED  PRIMARY  CARE  PROVIDER 
ASSESSMENTS  OF  PTSD  AND  MENTAL  HEALTH  CONDITIONS 
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BACKGROUND  AND  SIGNIFICANCE 

Military  personnel  returning  from  overseas  deployment  are  at  increased  risk  for  a  wide  range  of  physical 
and  mental  health  problems.  To  screen  for  such  difficulties  and  to  refer  Service  Members  (SMs)  in  need 
of  focused  clinical  evaluation  and  care,  the  military  departments  conduct  two  post-deployment  health 
risk  assessments.  The  Post-Deployment  Health  Risk  Assessment  (PDHA)  (DD  Form  2796,  see  Appendix  A) 
is  administered  as  close  to  the  redeployment  date  as  possible— within  30  days  before  SMs  depart  an 
overseas  assignment  or  within  30  days  after  they  return  to  home  station.  For  Reserve  Component 
members,  the  PDFIA  must  be  conducted  before  they  are  released  from  active  duty.  The  Post- 
Deployment  Health  Reassessment  (PDHRA)  (DD  Form  2900,  see  Appendix  B)  is  conducted  within  90  to 
180  days  of  redeployment.  Item  contents  and  clinical  procedures  for  the  PDFIA  and  PDFIRA  are  closely 
parallel.  Each  entails  multi-stage  processes  requiring  that  the  SM  complete  a  self-report  assessment  of 
physical  and  emotional  symptoms,  experiences  with  several  aspects  of  combat,  and  exposures  to  a 
variety  of  environmental  and  chemical  agents  while  in  the  combat  zone.  Following  this,  the  SM  is 
individually  evaluated  by  a  trained  health  care  provider  and  is  given  education  and  informational 
materials  relevant  to  his  or  her  concerns.  Flealth  care  providers  also  make  referrals  for  further 
evaluation  and  follow-up  treatment  on  the  basis  of  clinical  judgment. 

There  are  concerns  about  SM  under-reporting  of  mental  health  issues  on  the 
Post-Deployment  Health  Reassessment  (PDHRA) 

In  our  previous  study  (Bickman  et  al.,  2009),  anonymous  surveys  were  collected  from  SMs  completing 
the  PDFIRA  process.  A  substantial  minority  (10-14%)  of  SMs  admitted  to  underreporting  physical, 
emotional,  and  alcohol  use  problems  on  the  PDFIRA.  More  than  a  third  (39%)  of  SMs  agreed  that  they 
had  experienced  an  emotional,  alcohol,  stress,  or  family  problem  since  returning  from  deployment  or 
that  family  or  friends  had  suggested  they  seek  help  for  such  a  problem.  Flowever,  almost  half  (43%)  of 
these  SMs  did  not  report  any  such  problem  on  the  PDFIRA  form.  Further,  these  unreported  problems 
were  usually  not  uncovered  (i.e.,  documented)  by  the  health  care  provider  during  the  interview.  That  is, 
providers  documented  five  times  fewer  major  concerns  and  three  times  fewer  medical  referrals  for 
those  who  did  not  disclose  problems  on  the  assessment  form  versus  those  who  did  disclose. 

Reasons  for  under-reporting  could  include  concerns  about  stigma  or  barriers  to  care  (e.g.,  perceived  lack 
of  effective  treatments).  Floge  et  al.  (2004)  reported  that  only  half  of  those  who  screened  positive  for  a 
mental  disorder  sought  mental  health  care.  Furthermore,  SMs  who  screened  positive  for  a  mental 
health  problem  were  twice  as  likely  as  those  who  did  not  to  endorse  concerns  about  stigma  and  barriers 
to  care. 

We  found  that  measures  of  perceived  stigma  and  barriers  to  care  were  higher  for  SMs  who  reported  on 
an  anonymous  survey  that  family  or  friends  had  suggested  they  seek  help  or  confidentially  reported  an 
emotional,  alcohol,  stress,  or  family  problem  since  returning  from  deployment.  These  SMs  also  reported 
lower  satisfaction  with  the  PDFIRA  provider,  less  post-deployment  support  and  help  seeking,  and  less 
general  willingness  to  self-disclose  (Bickman  et  al.,  2009). 

The  provider  interview  is  an  important  opportunity  to  identify  previously 
unreported  behavioral  health  issues 

During  the  PDFIRA  process,  the  provider  interview  offers  the  opportunity  for  providers  to  reduce 
concerns  about  stigma  or  barriers  to  care.  Flowever,  audio  recordings  of  telephone  PDFIRA  interviews 
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that  were  coded  for  content  and  socio-emotional  exchange  suggest  there  is  substantial  room  for 
improvement,  especially  regarding  mental  health  issues.  We  found  that  providers  were  less  likely  to 
explore  behavioral  health  issues  than  physical  health  issues  (Bickman  et  al.,  2009).  Physical  health  was 
almost  always  mentioned,  regardless  of  SM  endorsement  on  the  self-report  (87%  v  84%),  but  behavioral 
health  topics  were  mentioned  more  when  SMs  endorsed  concerns  (64%)  than  when  no  concerns  were 
endorsed  (35%).  Furthermore,  we  found  that  education  related  to  behavioral  health  issues  was  provided 
in  only  14%  of  all  calls,  although  this  increased  to  24%  in  calls  where  a  medical  referral  was  given. 

Finally,  we  found  that  communication  strategies  to  elicit  more  self-disclosure  were  lacking.  For  example, 
providers  asked  five  times  more  closed-ended  than  open-ended  questions,  and  rapport  building 
statements  (e.g.,  empathy,  legitimation)  occurred  in  less  than  6%  of  calls. 

For  appraisal  processes  that  include  a  self-report  questionnaire  and  clinical  assessment,  like  the  PDFIRA, 
the  sensitivity  and  specificity  for  the  individual  components  have  not  been  established  (Rona,  Flyams,  & 
Wessely,  2005).  The  success  of  the  PDFIRA  process  in  helping  SMs  receive  further  evaluation  where 
warranted  depends  on  both  the  SM  and  the  provider.  For  example,  whether  the  SM  self-identifies  on 
the  self-report  depends  on  awareness,  willingness  to  disclose,  environment  specific  factors  (e.g., 
leadership  support),  and  understanding  of  the  questions  on  the  form.  The  provider  interview  adds  the 
opportunity  to  identify  SMs  in  need  of  assistance  based  not  only  on  the  self-report,  but  also  the 
provider's  evaluation  of  the  SM's  presentation  during  the  interview.  This  is  especially  useful  for  items 
where  SMs  are  reluctant  to  report  because  of  perceived  stigma,  such  as  with  mental  health  issues.  Yet, 
evidence  suggests  that  the  provider  interview  does  little  to  increase  sensitivity  of  the  process.  After 
accounting  for  the  number  of  problems  areas  endorsed  by  SMs,  provider  documented  concerns  made 
only  a  small  contribution  to  predicting  who  received  a  medical  referral.  The  number  of  problem  areas 
endorsed  by  the  SM  explained  20%  of  the  variance  in  medical  referrals;  adding  provider  major  concerns 
as  documented  on  the  PDFIRA  explained  an  additional  7%  of  variance.  While  this  leaves  a  large 
percentage  of  variance  unaccounted  for,  the  main  point  here  is  that  the  SM-reported  problems  are  the 
main  predictor  of  a  referral,  with  the  clinical  interview  as  documented  on  the  PDFIRA  adding  a  relatively 
small  contribution  (Bickman  et  al,  2009). 

Improved  provider  communication  skiiis  couid  enhance  the  quaiity  of  the  PDHRA 
interview 

Research  in  a  broad  range  of  areas  indicates  that  patient-provider  interactions  can  be  enhanced  by 
attention  to  training  in  interpersonal  communication  patterns  (e.g.,  active  listening).  Providers  who  have 
received  training  on  interpersonal  communication  skills  provide  more  medical  counseling  (Brown  et  al., 
2000),  elicit  more  information  and  concerns  from  the  patient  (Joos  et  al.,  1995;  Rao  et  al.,  2007; 
Langewitz  et  al.,  1998),  exhibit  greater  facilitative  communication  and  information  giving  (Kim  et  al., 
2002;  Rao  et  al.,  2007),  ask  more  open  ended  questions,  ask  patients  for  opinions  more  frequently,  give 
more  biomedical  information,  have  less  negative  affect  (Levinson  &  Roter,  1993),  show  improved  overall 
communication  skills  (Back  et  al.,  2007;  Roter  et  al.,  2004;  Flelitzer  et  al.,  2010;  Rao  et  al.,  2007;  Roter  et 
al.,  1995;  Fallowfield  et  al.,  2003),  and  receive  higher  patient  satisfaction  ratings  (Rao  et  al,  2007; 
Frosthom  et  al.,  2005). 

Patients  visiting  providers  who  have  received  training  in  interpersonal  skills  communicate  more  during 
the  interaction  (Brown  et  al.,  2000),  disclose  more  medical  and  psychosocial  information  (Brown  et  al., 
2000;  Levinson  &  Roter,  1993),  are  more  satisfied  with  the  provider  (Brown  et  al.,  2000),  perceive 
receiving  more  information  from  the  provider  (Joos  et  al.,  1995;  Rao,  2007),  and  report  reductions  in 
symptoms  and  impairment  (Wissow  et  al.  2008)  and  in  emotional  distress  (Roter  et  al.,  1995). 
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It  should  be  noted,  however,  that  most  of  the  research  in  this  area  has  been  conducted  with  general 
practitioners;  the  applicability  to  a  brief  assessment  interview  warrants  further  consideration. 


Characteristics  of  a  good  communications  training  intervention 

There  are  at  least  6  indicators  of  a  quality  communication  skills  training  (Maguire  and  Pitceathly,  2002): 

1.  Provide  evidence  of  current  deficiencies  in  communication,  reasons  for  them,  and  the  consequences 
for  patients  and  doctors 

2.  Offer  an  evidence  base  for  the  skills  needed  to  overcome  these  deficiencies 

3.  Demonstrate  the  skills  to  be  learned  and  elicit  reactions  to  these 

4.  Provide  an  opportunity  to  practice  the  skills  under  controlled  and  safe  conditions 

5.  Give  constructive  feedback  on  performance  and  reflect  on  the  reasons  for  any  blocking  behavior 

6.  Provide  ongoing  support  and  encouragement 

Intervention  intensity  is  also  important.  Many  effective  training  programs  are  moderate  to  high 
intensity,  involving  at  least  one  day  of  initial  training  (Fallowfield  et  al.,  2003;  Levinson  &  Roter,  1993; 
Rao  et  al.,  2007).  Shorter  trainings  are  often  not  effective  or  less  effective  (Cheraghi-Sohi  &  Bower, 

2008;  Levinson  &  Roter,  1993;  Joos  et  al.,  1995). 

Intensity  of  the  training  is  not  just  associated  with  length,  but  also  with  the  level  of  experiential  learning 
and  interactivity  of  training  strategies.  Indeed,  focusing  on  length  may  be  confounded  by  the  typical 
didactic  nature  of  shorter  trainings.  A  recent  review  indicates  that  didactic  training  (e.g.,  typical  CME 
workshops)  is  less  effective  than  mixed  didactive  and  interactive  workshops  for  improving  health  care 
provider  practice  and  health  care  outcomes  (Forsetiund  et  al.,  2009).  Some  specific  components  of 
successful  training  include  providing  the  evidence  base  for  the  suggested  skills,  the  use  of  role  play 
and/or  simulated  patients,  modeling  (i.e.,  positive  and  negative  examples),  and  allowing  participants  to 
explore  their  own  feelings  regarding  the  desired  skills  (Merckaert  et  al.,  2005;  Aspegren,  1999). 


PROPOSED  STUDY  CHANGES  FOR  APPROVAL  BY  MRMC 

The  current  study  is  consistent  with  the  original  aims  as  proposed  in  the  approved  award.  Flowever, 
there  are  modifications  to  the  timeline  and  design  due  to  delays  caused  by  intervening  events  as 
described  in  the  first  year  report.  Below  is  a  summary  of  the  two  primary  challenges  faced  by  the  team: 

1.  2009  NDAA  legislation  (Sec.  708)  mandated  substantial  revisions  to  the  health  risk  appraisal 
process  and  instituted  new  requirements  for  provider  training.  As  of  this  writing,  the  NDAA 
training  slides  are  available  online,  but  the  video  is  still  being  developed  (and  thus  unavailable 
for  review).  Further,  the  Army  is  currently  piloting  the  secondary  stage  screening  forms.  Our 
study  can  proceed  regardless  of  whether  the  new  NDAA  requirements  are  implemented  at  study 
sites  or  not.  Our  pilot  is  anticipated  to  occur  in  June-July  2011. 

2.  Substantial  delays  have  been  experienced  in  receiving  data  from  the  Armed  Forces  Flealth 
Surveillance  Center  (AFFISC)  required  by  Vanderbilt  and  Purdue  to  complete  secondary  analyses 
relevant  to  Aim  1.  A  data  use  agreement  (DUA)  was  signed  by  all  parties  on  14-JAN-lO;  however, 
as  described  in  detail  in  the  first  year  report,  Vanderbilt  did  not  receive  the  data  from  AFFISC 
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until  06-JAN-ll.  An  issue  has  arisen  concerning  linking  data  from  VA  and  AFHSC,  but  a  solution 
has  been  found  and  efforts  are  underway  to  begin  the  linking  process. 

Following  is  a  summary  of  the  proposed  modifications  to  the  previously  approved  Scope  of  Work  (SOW) 
timeline  and/or  tasks  for  review  and  approval  by  MRMC.  There  are  no  cost  changes  associated  with 
these  modifications. 

Proposed  Modifications  to  Aim  One  to  Be  Approved 

The  intent  of  the  tasks  for  Aim  1  remains  the  same,  to  determine  key  elements  of  a  training  program  for 
providers  conducting  deployment-related  assessments.  The  three  proposed  changes  are: 

1.  Eliminate  the  focus  groups  with  key  stakeholders  involved  in  the  PDFIRA  process.  The 
justification  for  this  change  is  to  allow  the  project  to  complete  data  collection  by  SEP-11.  As 
described  in  the  first  year  report,  focus  groups  were  not  conducted  during  the  first  year  of  the 
project  due  to  the  impact  of  the  evolving  nature  of  the  NDAA  Sec.  708  training.  We  believe  that 
the  thorough  literature  review,  informal  conversations  with  key  PDFIRA  personnel,  and  guidance 
by  the  Expert  Panel  are  sufficient  substitutes. 

2.  Incorporate  the  potential  effects  of  the  NDAA  Sec.  708  training  into  the  intervention  design  by 
making  pilot  training  content  consistent  with  (and  expanding  upon)  the  portion  of  the  NDAA 
online  training  slides  available  on  3-FEB-ll  that  addresses  the  therapeutic  rapport  between  SM 
and  provider  (particularly  slides  41-43  available  at 
http://fhpr.osd.mil/pdfs/NDAA%20FFIP  DFICC.pdf). 

3.  Extend  the  timeline  for  the  secondary  analysis  of  data  conducted  by  Vanderbilt  and  Purdue 
Universities  through  the  no-cost  extension  year  (to  SEP-12).  The  justification  for  this  change  is  to 
allow  adequate  time  for  analysis  and  interpretation  of  these  highly  complex  datasets.  To  the 
degree  possible,  any  results  will  be  used  to  inform  the  intervention  (Aim  2)  as  it  is  developed. 
Results  will  be  incorporated  into  the  final  report  to  inform  interpretation  of  the  results  of  new 
data  analyses  conducted  for  this  study. 

Proposed  Modifications  to  Aim  Two  to  Be  Approved 

The  intent  of  the  tasks  for  Aim  2  remains  the  same,  to  evaluate  the  feasibility  and  effectiveness  of  a  pilot 
training  program.  The  three  proposed  changes  are: 

1.  Simplify  the  research  design  to  the  interrupted  time  series  with  non-equivalent  comparisons  as 
described  in  the  remainder  of  this  document.  The  simplified  design  also  uses  survey  methods  to 
assess  outcomes  rather  than  intensive  coding  of  audiotaped  PDFIRA  interviews.  This  change 
allows  for  a  shorter  time  period  for  data  collection  and  fewer  providers  needed  for  minimum 
power  to  detect  medium  effect  sizes.  The  justification  for  this  change  is  to  allow  the  project  to 
complete  data  collection  by  SEP-11. 

2.  Incorporate  the  potential  effects  of  the  NDAA  Sec.  708  training  into  the  study  design  by:  (a) 
collecting  data  from  FFIP&R  on  provider  completion  of  the  online  NDAA  training  for  providers 
involved  in  the  study  (if  feasible);  and  (b)  incorporating  questions  about  NDAA-related 
implementation  in  study  measures  (e.g.,  the  PDFIRA  program  manager  interview,  the  provider 
background  form).  Even  though  providers  may  not  be  required  to  complete  NDAA  training  until 
after  Vanderbilt's  study  is  complete,  we  still  need  to  track  whether  providers  had  been  exposed 
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to  the  training  if  we  are  to  control  for  its  effects  during  the  study.  Exposure  to  the  training  may 
moderate  effects  due  to  our  intervention. 

3.  Extend  the  timeline  for  analysis  of  data,  interpretation  of  results,  and  report-writing  through  the 
no-cost  extension  year  (to  SEP-12). 


RESEARCH  AIMS 

Aim  1:  Determine  key  elements  of  and  short  term  impact  of  training  programs  for  deployment  related 
assessments.  The  focus  will  be  on  guidance  related  to  eliciting  more  candid  reporting  of  behavioral 
health  concerns,  identification  of  behavioral  health  concerns  that  warrant  referral  and  motivating  the 
SM  to  accept  a  referral  for  further  evaluation  and/or  treatment  for  behavioral  health  conditions  and 
concerns.  This  aim  will  be  accomplished  through  (a)  Expert  Panel  review  of  results  from  2007-2009  VU- 
FHP&R  collaboration  to  determine  criteria  for  clinical  competencies;  (b),  review  of  the  NDAA  Sec.  708 
training  to  assist  in  identifying  key  elements  for  training  interventions  relevant  to  content,  format,  and 
implementation  and,(c)  secondary  analysis  of  PDHRA  data  from  a  specifically  developed  database  that 
includes  provider  and  military  treatment  facility  (MTF)  identifiers  that  will  allow  identification  of 
variability  in  concerns  and  referrals  attributed  to  the  provider,  over  and  above  SM  self-reported 
problems.  In  addition,  Purdue  will  conduct  a  secondary  analysis  of  DoD  and  VA  data  to  identify  PDFIRA 
variables  associated  with  the  development  of  and  recovery  from  PTSD. 

Note  that  the  remainder  of  this  document  focuses  on  Aim  2  as  the  primary  study. 

Aim  2:  Evaluate  the  effectiveness  of  a  targeted  training  and  feedback  program  on  primary  care 
provider's  interview  and  clinical  communication  patterns  related  to  SM  behavioral  health  condition 
identification  and  referrals.  A  training  workshop  that  incorporates  experiential  learning  strategies  and 
evidence-supported  characteristics  of  high  quality  communication  training  programs  will  be  piloted  with 
a  group  of  approximately  10  providers  who  conduct  PDFIRAs  at  two  to  three  sites.  All  providers  at  the 
intervention  sites  who  agree  to  participate  in  the  study  will  participate  in  the  training.  As  an  interrupted 
time  series  design,  each  provider  will  serve  as  his/her  own  control  through  the  administration  of 
measures  and  collection  of  existing  data  sources  (e.g.,  PDFiRA)  for  a  time  period  prior  to  and  following 
the  training.  The  use  of  a  time  series  approach  will  allow  us  to  determine  the  influence  of  the 
communication  training  as  a  main  effect  as  well  as  account  for  threats  to  validity,  such  as  changes  that 
occur  over  time  independent  of  the  intervention. 

Implementation  will  be  measured  through  training  attendance  records,  evaluations  completed  by  the 
providers,  and  study  team  observations  and  recorded  notes  of  the  training.  Potential  moderating 
variables  will  be  measured  through  a  provider  background  form  (e.g.,  professional  background, 
demographics,  self-efficacy  in  patient-centered  communication),  a  PDFIRA  program  manager  interview 
(e.g.,  typical  PDFIRA  processes,  existing  training  programs,  etc.),  and  analysis  of  secondary  data  of 
electronic  records  (PDFIA,  provider  completion  of  the  NDAA  online  training  and  related  test  scores). 
Outcome  measurement  will  include  brief  post-PDFIRA  surveys  completed  by  the  SM  (anonymously)  and 
provider  immediately  after  the  PDFIRA  interview  and  an  analysis  of  secondary  data  (including  electronic 
records  for  PDFIRA  and  health  care  encounters). 
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In  order  to  further  control  for  threats  to  external  validity,  we  will  also  passively  collect  data  from  non¬ 
equivalent  comparison  sites.  In  spite  of  the  short  timeframe  for  data  collection  (two  months), 
maturation  (the  passage  of  time  not  specific  to  the  event)  and  history  (events  that  occur  between  the 
first  and  second  measurements)  are  still  threats  whose  potency  can  be  reduced  by  the  inclusion  of  the 
comparison  sites.  Given  the  additional  time  and  logistics  needed  and  questionable  feasibility,  and  to 
reduce  cost  and  burden  to  AFHSC  and  potential  comparison  sites,  we  will  not  perform  active  data 
collection  at  specific  sites,  but  will  instead  request  data  for  the  study  time  period  (PDHA,  PDHRA,  HCE, 
and  NDAA  training  completion  and  test  score  data)  for  all  Army  installations  and  then  choose 
appropriate  comparison  sites  based  on  similarity  to  the  intervention  sites  (see  Site  Selection,  below). 

This  procedure  will  eliminate  the  need  to  create  a  separate  memorandum  to  the  Surgeon  General  of  the 
Army  for  recruitment  of  comparison  sites.  All  data  collection  at  comparison  sites  will  be  passive,  and  will 
take  place  over  approximately  the  same  time  period  as  data  collection  for  the  pilot. 

Research  questions  and  hypotheses  related  to  Aim  2  include: 

1.  Can  a  brief  intervention  to  enhance  communication  skills  be  implemented  in  the  field? 

a.  Any  increase  in  the  length  of  the  PDHRA  interview  is  within  an  acceptable  range. 

b.  Key  personnel  (i.e.,  participating  providers  and  program  managers)  find  the  intervention 
to  be  relevant  to  their  work  and  acceptable. 

2.  Will  this  intervention  help  providers  use  the  interview  as  an  opportunity  to  identify  SMs  in  need 
of  assistance  for  behavioral  health  problems? 

a.  Increased  provider  concerns  and  referrals  for  behavioral  health  issues  documented  on 
the  PDHRA. 

b.  Higher  ratings  of  SM  self-reported  disclosure;  intent  to  comply  with  referral;  and  ratings 
of  provider  patient-centered  communication. 

c.  Higher  ratings  of  provider-reported  elicitation  of  behavioral  health  concerns. 

INTERVENTION  DESIGN:  COMMUNICATION  TRAINING  WORKSHOP 
Format 

VU  will  arrive  at  the  site  2-3  days  prior  to  the  intervention  to  collect  data  pre-training.  The  trainer  will 
arrive  on  site  on  the  3'^'^  day  to  deliver  the  training  workshop.  VU  would  remain  on  site  2-3  days  post 
intervention  to  collect  post-training  data. 

The  workshop  content  will  be  tailored  to  the  PDHRA  encounter  and  will  be  informed  by  the 
communications  skills  presented  in  the  NDAA  training,  as  well  as  published  literature  on  best  practices  in 
patient  centered  communication  techniques  (see  Table  1,  below). 

The  workshop  will  last  approximately  4  hours  and  will  include  established  quality  techniques,  such  as 
establishing  need  for  training,  eliciting  provider  experiences/  frustrations,  introducing  and 
demonstrating  skills,  group  discussion,  and  providing  the  opportunity  to  practice  and  receive  feedback. 
The  practice  will  occur  in  the  form  of  either  role  play  or  interaction  with  a  standardized  patient  (SP). 

TBD  -  Possibility  of  incUvidualized  feedback 
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Providing  individualized  feedback  and  consultation  is  an  established  method  for  enhancing 
training  effects.  This  could  be  in  the  form  of  a  follow-up  consultation  by  telephone  to  review 
actual  cases  with  providers  and  their  experience  in  implementing  communication  skills.  A  second 
option  is  to  have  providers  interact  with  a  standardized  patient  as  part  of  the  training,  and  to 
receive  feedback  on  the  interaction  from  both  the  SP  and  the  trainer.  Because  it  is  time 
consuming,  this  interaction  would  take  place  separately  for  each  provider  and  would  add  one 
hour  to  the  training  per  provider  (5  hours  total). 


Topics  Covered 

While  the  specifics  of  training  content  are  still  being  developed,  we  have  identified  the  communication 
behaviors  that  the  training  will  aim  to  improve.  These  are  divided  into  "Context-Free"  and  "Context- 
Specific"  behaviors.  By  context-free,  we  mean  provider  communication  behaviors  that  do  not  apply  to 
any  specific  area  of  the  PDHRA,  but  rather  are  viewed  as  consistent  with  a  patient-centered  approach. 

By  context-specific,  we  mean  provider  communication  behaviors  that  are  specific  to  the  PDHRA  process. 
The  purpose  of  the  PDHRA  is  to  increase  SM  access  to  care  where  warranted  and  provide 
documentation  of  deployment-related  concerns.  Published  material  available  on  pdhealth.mil  and 
elsewhere  states  four  primary  objectives  for  the  PDHRA:  (1)  Clarify  and  confirm  SM  responses  on  DD 
Form  2900;  (2)  Educate  SMs  about  concerns,  healthcare,  and  treatment  options;  (3)  Conduct  a  risk 
assessment;  and  (4)  Make  referrals  for  further  evaluation  where  warranted.  We  intend  to  target  several 
provider  communication  behaviors  that  are  consistent  with  the  patient-centered  approach  and  that 
expand  upon  these  four  PDHRA  objectives.  Table  1  summarizes  both  context-free  and  context-specific 
target  behaviors. 


Table  1:  Training  Content  Summary 


Context-Free  Communication  Behaviors 

Behavior  Targeted 

Example 

Increasing  use  of  open-ended  questions* 

"What  symptoms  are  you  having  right  now?"** 

Listening 

Decrease  in  ratio  of  provider  talk  to  SM  talk 

Expressing  empathy 

"That  would  be  depressing." 

Showing  concern 

"I'm  glad  it  worked  out . . . 

Providing  reassurance 

"There  are  effective  treatments  for  that." 

Legitimizing  statements 

"It  doesn't  get  any  easier." 

Asking  SM  opinion 

"Do  you  want  to  be  seen  for  that?" 

Active  listening 

Back  channeling  to  indicate  interest,  e.g.  "mmm"  or  "Tell 
me  more." 

Making  partnership  statements 

"1  can  get  that  information  for  you." 
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Table  1:  Training  Content  Summary  (continued) 


Context-Specific  Communication  Behaviors 

Behavior  Targeted 

Example 

Check  SM  understanding  of  PDHRA  purpose  and 
address  concerns  about  disclosure. 

What's  your  understanding  about  what  you're  doing  here 
today? 

Partnering  in  PDHRA  process 

Explaining  the  process,  what  the  provider  is  doing  with  the 
form  on  the  computer,  what  they  mark  down. 

Specifically  ask  open-ended  questions  about 
general  well-being  at  the  beginning  of  the 
interview  including  psychosocial  issues  related  to 
reintegration,  PTSD/depression,  relationships, 
and  alcohol  use  regardless  of  what  SM  marked 
on  DD  Form  2900 

•  What  concerns  do  you  have  that  1  can  help  you  with 
today? 

•  How  are  things  going  since  you  returned? 

•  Everyone  goes  through  an  adjustment  coming  home. 

How  is  it  going  for  you? 

•  Now  we've  talked  about  your  physical  health  problems. 
What  about  other  concerns  related  to  adjusting  to 
being  back  home,  like  feelings  of  being  worried  or  sad 
or  having  trouble  in  relationships?" 

Ask  specific  questions  of  all  SMs  who  report  a 
problem 

•  Ask  if  SM  has  received  treatment  or  is  in  treatment 

•  Ask  if  satisfied  with  treatment  or  feels  need  for  further 

treatment 

For  all  SMs  regardless  of  whether  referral  is 
warranted,  provide  brief  statements  to  legitimize 
common  reintegration  concerns 

Many  soldiers  have  ups  and  downs  adjusting  to  being  back. 

For  all  SMs,  give  brief  counseling  on  self-care  and 
self-referral  that  can  be  accessed  any  time 

You  can  always  talk  to  a  chaplain  or  make  an  appointment 
on  your  own  with  your  primary  care  provider.  (Note:  could 
also  refer  to  websites  and  other  resources) 

For  all  SMs  who  warrant  a  referral,  elicit  SM 
reactions  to  problem  identification/referral 
recommendation  and  address  concerns/barriers 

I'd  like  to  recommend  a  referral  for  that;  how  does  that 
sound  to  you? 

For  all  SMs  who  warrant  a  referral,  provide  brief 
education  on  treatment  effectiveness  for  mental 
health  problems. 

There  are  effective  treatments  for  that.  (Note:  could  also 
refer  to  resources  in  NDAA  Sec.  708  training  slides  if 
providers  are  already  familiar  with  it). 

For  all  SMs  who  warrant  a  referral,  check  SM 
understanding  of  how  to  achieve  referral 

Do  you  know  how  to  make  the  appointment  for  that?  or 
any  statement  explaining  the  next  step  in  SRP. 

Building  therapeutic  alliance  and  bridging  of 
social  distance 

•  Statements  that  acknowledge  cultural  differences  like 
civilian  provider,  deployment  experience,  leadership 
support. 

•  Thanking  SM  for  service 

*  Close-ended  questions  (e.g.,  "Have  you  been  screened  for  PTSD?")  and  checks  for  understanding  (e.g.,  "I  see  that 
you  were  in  an  explosion")  will  not  necessarily  decrease,  because  these  are  indicative  of  the  PDHRA. 

** Examples  in  quotation  marks  are  actual  provider  utterances  from  de-identified  audio-recordings  collected  during 
Vanderbilt's  previous  research  (Bickman  et  al.,  2009). 
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AIM  2  DESIGN  AND  METHODOLOGY 

The  design  is  an  interrupted  time  series  with  non-equivalent  comparison  sites.  The  intervention  will  take 
place  during  site  visits  to  2-3  installations.  The  comparison  sites  are  labeled  non-equivalent  because  we 
are  not  randomly  assigning  comparison  and  intervention  sites  to  the  intervention.  The  inclusion  of 
comparison  sites  will  allow  for  measurement  of  common  threats  to  validity  (e.g.,  Army-wide  changes  in 
PDHRA  processes  that  co-occur  with  the  intervention).  The  interrupted  time  series  data  will  be  collected 
before  and  after  the  intervention  so  each  provider  serves  as  his/her  own  control. 

At  all  sites,  we  will  be  collecting  previously  existing  data  (e.g.,  passive  data  collection)  related  to  PDHA, 
PDHRA,  health  care  encounter  information,  and  NDAA  training.  At  the  intervention  sites,  we  will  be 
actively  collecting  data  through  survey  methods  and  qualitative  methods  (interviews  and  observation) 
as  described  further  below. 


Site  Selection 

This  study  targets  Army  installations.  The  intervention  sites  will  be  selected  based  on  number  of 
providers  and  PDHRA  flow  through.  Also,  sites  that  previously  expressed  interest  in  participating 
(Campbell,  Riley,  Benning,  Carson)  will  be  considered.  This  introduces  the  possibility  of  a  "volunteer 
effect"  creating  systemic  bias.  However,  because  we  are  only  including  2-3  intervention  sites, 
generalizability  will  be  limited  in  any  case.  In  addition,  for  a  pilot,  demonstrating  generalizability  is  less 
important  than  maximizing  the  chances  of  finding  an  effect.  Using  sites  that  have  previously  expressed 
interest  is  likely  to  result  in  higher  levels  of  leadership  and  provider  cooperation,  which  will  increase 
chances  of  detecting  positive  results.  After  a  pilot  demonstrates  feasibility  and  effectiveness,  questions 
of  whether  the  intervention  is  generalizable  could  be  addressed  by  conducting  an  evaluation  of 
implementation  and  effectiveness  of  the  intervention  at  a  broader  range  of  installations. 

The  non-equivalent  comparison  sites  will  be  selected  from  the  Army-wide  dataset.  The  data  will  be 
evaluated  at  the  installation  level  for  PDHRA  flow  through,  number  of  providers,  and  types  of  units. 
Then  comparison  sites  will  be  selected  according  to  comparability  with  the  intervention  sites  based  on 
these  criteria. 

Study  Sample 

Table  2  shows  the  number  of  expected/required  sites  and  participants.  Previously  identified  potential 
study  sites  and  the  approximate  number  of  available  providers  at  each  are  shown  in  Table  3. 
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Table  2.  Number  of  expected/required  participants. 


Participant 

Number  required/expected  to  participate 

Intervention  Sites 

Comparison  Sites* 

Site 

2-3 

At  least  2-3 

Program  Manager 

2-3  (One  per  site) 

n/a 

Providers 

12  total 

At  least  30 

SMs 

80  minimum  per  provider,  but 
ideally  as  many  as  possible** 

20  minimum*** 

*Note  that  data  collection  at  comparison  sites  will  not  require  active  participation  since  only  passive  data 
collection  will  occur. 

**  More  SMs  would  be  lead  to  smaller  standard  errors  and  narrower  confidence  intervals  (i.e.,  more 
precision). 

***  20  SMs  would  detect  a  small  intraclass  correlation  (ICC).  A  significant  ICC  would  indicate  that  providers 
differ  from  each  other  rather  than  offering  a  uniform  standard  of  care. 


Table  3.  Previously  identified  installations  and  number  of  available  providers. 


Potential  Intervention  Sites 

Number  of  Providers 

A 

4-6 

B 

3-4 

C 

16 

D 

unknown 

Recruitment 

Sites 

Intervention  sites  will  be  recruited  via  a  memorandum  to  the  Surgeon  General  of  the  Army  describing 
study  events  in  detail  and  requesting  the  nomination  of  sites  for  participation.  The  memorandum  will  be 
prepared  in  cooperation  with  FHP&R.  Because  only  passive  data  collection  will  take  place  at  comparison 
sites,  no  memorandum  to  the  Surgeon  General  of  the  Army  will  be  required  for  their  nomination. 

Program  Managers  and  Providers 

The  responsibilities  of  Program  Managers  and  providers  during  the  study  will  be  described  in  the 
memorandum  sent  to  the  Army  Surgeon  General,  and  appropriate  site  personnel  will  inform  these 
individuals  of  the  tasks  involved  in  participation. 

All  study  procedures  will  be  submitted  to  the  Vanderbilt  and  MRMC  IRBs  for  approval  prior  to 
implementation.  Separate  IRB  protocols  will  be  prepared  for  the  Program  Manager  interview,  and  the 
provider  training  and  survey  completion.  The  Program  Manager  protocol  is  expected  to  be  approved  as 
exempt  (i.e.,  not  requiring  annual  review  by  IRB)  under  45  CFR  46.101(b)(2)  (non-identifiable  data  and 
minimal  risk  to  subjects).  The  provider  training  protocol  will  be  submitted  as  expedited  and  informed 
consent  will  be  obtained  from  all  participants. 


SMs 
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It  is  expected  that  SM  survey  procedures  will  be  submitted  as  exempt  under  45  CFR  46.101(b)(2).  All 
SMs  completing  the  PDHRA  process  during  VU's  site  visit  will  be  eligible  to  complete  the  survey.  We 
expect  recruitment  will  occur  during  the  pre-briefing  that  SMs  are  typically  given  prior  to  starting  the 
PDHRA  process.  The  recruitment  script  will  be  delivered  by  a  VU  researcher  with  a  short  introductory 
statement  by  the  site  personnel  giving  the  pre-briefing.  SMs  agreeing  to  complete  the  survey  would  be 
instructed  to  do  so  after  completing  the  PDHRA  interview. 

Measures 

Research  measures  used  in  this  study  fall  into  three  categories:  1)  Existing  data  sources,  2)  Vanderbilt- 
developed  quantitative  measures,  and  3)  Vanderbilt-developed  qualitative  measures.  The  sections 
below  briefly  describe  the  content  and  administration  details  of  each  measure.  For  a  description  of  data 
management  issues  associated  with  these  datasets,  including  a  summary  of  how  different  datasets  will 
be  linked  to  each  other,  see  the  Analysis  Plan  below. 

Existing  Data  Sources 

These  measures  come  from  pre-existing  sources  and  do  not  require  any  additional  time  commitment 
from  providers  or  SMs  (see  Table  4). 


63 


Annual  Report:  Contract  #W81XWH-09-2-0172 


Table  4:  Existing  Data  Sources 


Data  Source 

Construct 

Respondent 

Collected  from 

Frequency  and 
Collection  Period 

Intervention 

Sites 

Comparison 

Sites 

PDHRA 

•  SM  self- 
reported 
symptoms 

•  Provider 

documented 

concerns 

•  Provider 

documented 

referrals 

•  Demographic 
variables 

SM  and  Provider 
(includes 
provider  ID) 

X 

X 

All  PDHRAs  completed 
during  the  study  time 
period:  4  weeks  before 
and  4  weeks  after 
intervention  (8  weeks 
total) 

All  pre-existing  PDHRAs 
associated  with  the 

PDHRAs  collected  in  the 

timeframe.  These  data 
will  include  a  unique 
StudylD  for  each  SM. 

PDHA 

•  SM  self- 
reported 
symptoms 

•  Provider 

documented 

concerns 

•  Provider 

documented 

referrals 

•  Combat 

exposure 

SM  and 

Provider 

X 

X 

All  pre-existing  PDHAs 
associated  with  the 

PDHRAs  collected  in  the 

timeframe  described 

above.  These  data  will 
include  a  unique  StudylD 
for  each  SM. 

Health  Care 

Encounter 

(HCE) 

•  Dates  of 
encounters, 
admissions, 
discharges 

•  Setting  of 
encounters 

ICD-9  Code 

(Diagnosis) 

•  CPT  Code 
(Service 
provided) 

n/a 

X 

X 

All  pre-existing  HCE 
associated  with  the 

PDHRAs  collected  in  the 

timeframe  described 

above  PLUS  an  additional 

six  weeks  after  the 

PDHRA.  These  data  will 
include  a  unique  StudylD 
for  each  SM. 

NDAA 
training 
completion 
and  final 

score* 

•  Date 
completed 

•  Post-test 

score 

Provider 

X 

X 

One  time  4  weeks  after 

intervention.  These  data 

will  include  fields  for 
provider  service,  rank, 
name,  type,  and  duty 
station. 
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Table  4:  Existing  Data  Sources  (continued) 


Data  Source 

Construct 

Respondent 

Collected  from 

Frequency  and 
Collection  Period 

Intervention 

Sites 

Comparison 

Sites 

DEERS 

•  Education 

level 

•  Component 
at  form 
completion 

•  Race  and 
ethnicity 

•  Unit 

identification 
code  (UlC). 

SM 

X 

X 

All  data  associated  with 

the  PDHRAs  collected  in 

the  timeframe  described 

above.  These  data  will 
include  a  unique  StudylD 

Army 

(MEDPROS) 

•  Provider  ID 

•  Location  ID 

n/a 

X 

X 

One  time  for  all 

intervention  and 
comparison  sites 

Intervention 

installations 

•  Provider  ID 

•  Location  ID 

•  Form 
completion 
date 

•  Form  version 

n/a 

X 

One  time  4  weeks  after 

intervention.  These  data 
will  include  a  unique 
study  ID  for  each  SM 

*  We  plan  to  receive  these  data  from  FHP&R.  However,  if  this  is  not  possible  due  to  security  restrictions,  we  will 
ask  individual  providers  for  these  data. 


65 


Annual  Report:  Contract  #W81XWH-09-2-0172 


Quantitative  Measures 

There  quantitative  measures  include  written  surveys  to  be  completed  by  providers  or  SMs.  All  are  brief 
instruments  which  will  require  a  minimal  amount  of  time  for  respondents  to  complete.  These  measures 
are  summarized  briefly  in  the  table  below. 


Table  5:  Quantitative  Measures 


Data  Source 

Respondent 

Time  to 
Complete 

Constructs 

Collected  from 

Frequency 

and 

Collection 

Period 

Intervention 

Sites 

Comparison 

Sites 

SM  Post  - 

PDHRA 

satisfaction 

survey 

SM 

5-10 

minutes 

•  Reported 
disclosure  of 
mental  health  (MH) 
symptoms 

•  Intent  to  comply 
with  referral  (if 
given) 

•  Attitudes  to 

disclosure  and 
help-seeking 

•  Concerns  about 
barriers  and  stigma 

•  Previous  help¬ 
seeking 

•  Self-reported  MH 

concerns 

•  Satisfaction  with 
provider 

•  Rating  of  provider 
patient-centered 
communication 

X 

All  SMs 

completing 

PDHRAs 

within  2-3 
days  pre- 
and  post¬ 
intervention 
(4-6  days 
total) 
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Data  Source 

Respondent 

Time  to 
Complete 

Constructs 

Collected  from 

Frequency 

and 

Collection 

Period 

Intervention 

Sites 

Comparison 

Sites 

Provider 

post-PDHRA 

satisfaction 

survey 

Provider 

30 

seconds 

Providers  will  use  a 
Likert  Scale  to  report 
on  3  items: 

•  Whether  SM 
reported  MH 
symptoms  during 
interview  that  were 
not  reported  on  DD 
Form  2900 

•  Degree  to  which 
provider  believes 

SM  accurately 
reported  all  MH 
symptoms  during 
interview 

•  Whether  provider 
believes  SM  could 

benefit  from 

further  evaluation 
for  MH  symptoms 

X 

Completed 
after  each 

PDHRA 

interview 

conducted 

within  2-3 
days  pre- 
and  post¬ 
intervention 
(4-6  days 
total) 
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Table  5:  Quantitative  Measures  (continued) 


Data  Source 

Respondent 

Time  to 
Complete 

Constructs 

Collected  from 

Frequency 

and 

Collection 

Period 

Intervention 

Sites 

Comparison 

Sites 

Provider 

background 

survey 

Provider 

5 

minutes 

•  Demographic 
background 

•  Professional 
experience 

•  PDHRA  experience 

•  Self-efficacy, 
knowledge,  and 

attitudes  relevant 

to  patient-centered 

communication 

X 

One  time 
prior  to 
intervention 

Provider 

post 

intervention 

evaluation 

Provider 

10 

minutes 

•  Self-efficacy  and 
knowledge  relevant 
to  patient-centered 
communication 

•  Satisfaction  with 
training 

X 

One  time 

after 

intervention 
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Qualitative  Measures 

Three  qualitative  measures  will  also  be  used  during  the  study  and  will  be  administered  at  intervention 
sites  by  members  of  the  Vanderbilt  research  team. 

Program  Manager  Interview 

Before  the  communication  training  takes  place,  a  member  of  the  Vanderbilt  research  team  will  conduct 
a  twenty  minute  face-to-face  or  telephone  interview  with  the  Program  Manager(s)  at  each  intervention 
site.  This  interview  will  be  semi-structured  in  nature;  set  questions  will  be  asked  of  each  respondent, 
and  following  initial  responses,  interviewers  will  use  pre-developed  prompts  to  probe  for  more  detailed 
information.  Interviewers  will  take  notes  during  the  interview  but  will  also  use  digital  audio-recorders  to 
ensure  greater  accuracy  and  capture  of  detail.  Topically,  the  program  manager  interview  will  focus  on 
several  key  areas  of  interest  related  to  the  PDHRA  process:  general  PDHRA  background,  PDHRA 
implementation,  SM  pre-briefing  and  education,  command  support,  referral  processes  at  the 
installation,  background  and  training  of  providers  conducting  PDHRA  assessment,  utilization 
management  and  reporting,  and  general  barriers  and  facilitators.  Program  managers  at  intervention 
sites  will  also  be  asked  to  provide  feedback  on  the  intervention's  feasibility  and  effectiveness  in  a  second 
semi-structured  interview  to  take  place  after  the  intervention. 

Observation  of  Training  Sessions 

Members  of  the  Vanderbilt  research  team  will  observe  and  take  notes  on  trainings  at  each  intervention 
site.  In  order  to  improve  the  accuracy  of  recording  and  to  help  standardize  the  observations  of  multiple 
observers,  a  written  observation  guide  will  be  developed  and  used  during  all  trainings.  Training  sessions 
will  also  be  video  recorded  to  ensure  accuracy  of  observations. 

Observation  of  Time  for  Completion  of  PDHRA  Interviews 

Members  of  the  Vanderbilt  research  team  will  use  time-sampling  techniques  to  gather  data  on  duration 
of  the  PDHRA  interviews  at  each  intervention  site.  A  written  observation  sheet  will  be  developed  and 
used  to  collect  data. 

Analysis  Plan 
Data  Management 

The  PDHA,  PDHRA,  DEERS  variables,  and  health  care  encounter  (HCE)  data  will  be  received  from  AFHSC 
in  an  electronic  format.  The  NDAA  completion  data  will  be  received  in  an  electronic  format  from  FHP&R 
(if  feasible).  The  SM  and  provider  surveys  and  written  measures  associated  with  the  training  workshop 
will  be  collected  with  paper  and  pencil  during  site  visits.  CEPI  has  a  formal  data  management  workflow 
used  for  many  ROl-scale  projects  (Smith,  Breda,  Simmons,  Lambert  &  Bickman,  2009).  Raw  data  are 
captured  in  the  most  convenient  source,  e.g.,  double-entry  Microsoft  (MS)  Access  databases  and  files 
from  various  sources.  Data  are  then  arranged  in  orderly  hierarchy  on  VU's  Windows  server,  which  has 
daily  tape  backups  and  daily  security  checks  by  the  VU  network  manager  and  senior  technicians. 
Quantitative  data  are  then  exported  into  SAS  data  sets,  either  by  directly  reading  by  SAS  or  export  by 
software.  Then  an  array  of  SAS  programs  are  written  to  clean,  label,  and  transform  the  raw  data,  mark 
missing  values,  and  enforce  consistent  statistical  coding  (e.g.,  no-yes  0-1).  Finally,  the  SAS  data  sets  are 
merged  into  analytic  files  that  are  either  wide  (one  line  per  participant)  or  tall  (multiple  lines  per 
participant  for  repeated  measures  analysis). 
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Linking  Procedures 

Note  that  all  linking  procedures  are  consistent  with  existing  procedures  used  in  the  previous  Vanderbilt 
evaluation  and  in  retrieval  of  secondary  analysis  datasets  related  to  Aim  1  of  this  study  to  facilitate  ease 
of  use  for  AFHSC. 

The  linking  procedures  described  below  are  shown  graphically  in  Figure  1  on  the  next  page. 

Pre-existing  data  sources 

All  pre-existing  data  sources  except  NDAA  scores  (PDFIA,  PDFIRA,  FICE,  and  DEERS  variables;  see  Table  4) 
will  be  linked  because  AFFISC  assigns  each  case  (i.e.,  SM)  a  unique  study  ID  in  place  of  the  SSN. 
Vanderbilt  will  receive  these  data  de-identified  with  only  the  study  ID.  If  feasible,  the  NDAA  data  will  be 
obtained  from  FFIP&R  and  will  include  provider  name,  which  can  be  used  by  VU  to  link  these  data  to  the 
other  pre-existing  data  (provider  ID  from  the  Army  is  in  the  format  firstname.lastname).  If  FFIP&R 
cannot  provide  these  data  we  will  ask  individual  providers  from  study  sites  for  the  completion  date  and 
score. 

SM  survey  and  PDHRA 

All  SMs  who  agree  to  take  a  survey  will  be  asked  to  provide  their  birth  date,  initials,  branch  of  service, 
and  pay  grade  on  a  card  stapled  to  the  survey.  Each  card  will  be  printed  with  a  unique  serial  number, 
which  will  also  be  printed  on  the  survey.  The  cards  will  be  separated  from  the  surveys  and  sent  to 
FFIP&R  by  a  designated  individual  on  site  who  is  not  associated  with  Vanderbilt.  The  cards  will  then  be 
retrieved  by  an  outside  data  entry  company  which  will  enter  the  data  into  a  spread  sheet  and  return  the 
file  to  FFIP&R  via  email.  This  spreadsheet  will  be  sent  to  the  epidemiologist  at  AFFISC  who  has  access  to 
PDFIRA  files  and  who  will  pull  the  existing  data  for  this  study  (see  Table  4).  The  card  data  will  be  used  by 
the  epidemiologist  to  identify  the  subset  of  PDFIRAs  that  correspond  to  the  surveys.  AFFISC  will  assign 
each  record  (i.e.,  SM)  in  the  data  set  a  unique  study  ID  which  will  be  used  to  link  the  SM  survey  data  and 
the  existing  data.  After  all  identifying  information  has  been  removed  Vanderbilt  will  be  sent  the  file 
containing  the  unique  study  ID  and  corresponding  survey  ID.  Vanderbilt  will  maintain  the  hard  copies  of 
the  SM  surveys  from  site  visits,  which  will  be  labeled  with  the  survey  ID,  but  contain  no  identifying 
information.  Thus,  Vanderbilt  will  know  which  SM  Surveys  and  PDFIRAs  come  from  the  same  SM  but  will 
not  at  any  time  have  access  to  any  information  that  can  identify  SMs. 

Provider  and  SM  post-PDHRA  satisfaction  surveys 

The  SM  and  provider  satisfaction  surveys  will  be  linked  directly  to  each  other  with  a  unique  serial 
number  printed  on  each  pair  of  surveys.  The  corresponding  SM  and  provider  surveys  will  be  stapled 
together  and  detached  by  the  provider  just  before  completion.  The  surveys  will  be  collected  by  VU 
during  the  site  visit.  Thus,  it  will  be  possible  to  assess  the  impact  of  the  intervention  for  each  provider 
by  examining  the  corresponding  PDFIRA,  SM  survey,  and  provider  survey  for  each  PDFIRA  interview. 

Provider  surveys 

Providers  will  be  asked  to  indicate  their  provider  ID/name  with  a  check  box  on  each  survey  they 
compete.  In  our  previous  study  (Bickman  et  al.,  2009)  only  75%  of  SM  surveys  were  able  to  be  linked  to 
PDFIRAs  following  the  card  separation  procedures  described  above.  In  the  event  of  a  broken  link,  having 
provider  ID  and  name  on  the  provider  survey  will  allow  us  to  identify  each  provider's  surveys. 
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Figure  1.  Diagram  of  linking  procedures  for  all  data  elements.  Grey  boxes  indicate  linking  variables.  Double  lines  indicate  critical  links. 
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Expected  Data  Requests 

Data  requests  will  be  submitted  for  the  pre-existing  data  sources  described  in  Table  4.  The 
timeline  for  these  data  requests  is  described  below  for  each  organization  providing  data. 

Army 

The  data  request  to  the  Army  will  be  an  amendment  to  the  existing  data  request  to  include  data 
for  the  intervention  (i.e.,  for  SMs  who  completed  PDHRAs  within  4  weeks  before  and  after  the 
intervention).  This  amendment  will  be  submitted  as  soon  as  possible,  but  no  later  than  four 
weeks  after  the  intervention. 

AFHSC 

The  data  request  to  AFHSC  will  require  a  new  data  use  agreement  (DUA)  that  is  currently  being 
drafted.  The  data  request  will  be  submitted  as  soon  as  possible,  but  no  later  than  10  weeks 
after  the  intervention  (this  will  allow  for  six  weeks  of  post-PDHRA  HCE  for  PDHRAs  completed 
four  weeks  after  the  intervention) 

FHP&R 

If  feasible,  FHP&R  will  provide  NDAA  test  scores  and  completion  dates,  and  because  FHP&R  is 
the  co-sponsor  in  this  award,  they  will  submit  this  request.  This  request  will  be  submitted  as 
soon  as  possible,  but  no  later  than  4  weeks  after  the  intervention. 

Intervention  Sites 

The  specific  sites  for  the  intervention  have  not  been  selected  yet,  so  the  data  request 
procedures  are  unknown.  However,  the  data  request  will  be  submitted  as  soon  as  possible,  but 
no  later  than  four  weeks  after  the  intervention. 

Testing  Hypotheses 

The  analyses  are  organized  around  the  study's  specific  questions,  as  shown  in  Table  6. 

Table  6.  Specific  Questions  and  Analytic  Plan 


Specific  Question 

Design 

IVs 

DVs 

Can  a  brief  intervention  to  enhance  communication  skills  be  implemented  in  the  field? 

Any  increase  in  the  length 
of  the  PDHRA  interview,  is 
within  an  acceptable 
range. 

Mean  comparison  of  pre- 
and  post-  intervention 
interview  length  and  opinion 
of  provider 

Whether  the 

interview  or 

response  was 
pre-  or  post¬ 
intervention 

Mean  length  of 
interview  before 
and  after  training 
Duration  of 

interview  that  is 
acceptable 

Participating  providers  find 
the  intervention  to  be 

relevant  to  their  work  and 
acceptable. 

Mean  comparison  of  pre- 
and  post-  intervention 
responses 

Whether  the 

response  was 
pre-  or  post¬ 
intervention 

Change  in  self- 
efficacy  in  eliciting 
mental  health 

concerns  and 
interpersonal 
communication 

Satisfaction  with 
training  (post 
only) 
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Table  6.  Specific  Questions  and  Analytic  Plan  (continued) 


Specific 

Question 

Design 

IVs 

DVs 

Will  this  intervention  help  providers  use  the  interview  as  an  opportunity  to  identify  SMs  in  need 
of  assistance  for  behavioral  health  problems? 

Increased 
provider 
concerns  and 

referrals  for 

mental  health 

issues 

documented 

on  PDHRA 

Interrupted  time 
series  within  the 

intervention 

providers 

•  Time  from  the 

start  of  the 

observational 

period 

•  Whether  the 
observation  is  pre- 
or  post¬ 
intervention 

•  Time  since  the 

intervention 

•  Number  and  type  of  provider 
documented  concerns 

•  Presence  of  provider 
documented  medical  referral 

•  Presence  of  provider 
documented  mental  health 

referral 

Mean  comparison 
of  intervention 
group  and  non¬ 
equivalent 
comparison  group 

•  Whether  the 

observation  is 

from  the 

intervention  or 

comparison 

group 

•  Number  and  type  of  provider 
documented  concerns 

•  Presence  of  provider 
documented  medical  referral 

•  Presence  of  provider 
documented  mental  health 

referral 

Higher  ratings 
of  SM  self- 
reported 
disclosure; 
intent  to 
comply  with 
referral;  SM 
ratings  of 
provider 
patient- 
centered 

communication 

Interrupted  time 
series  within  the 

intervention 

providers 

•  Time  from  the 

start  of  the 

observational 

period 

•  Whether  the 
observation  is  pre- 
or  post¬ 
intervention 

•  Time  since  the 

intervention 

•  SM-reported  disclosure 

•  Intent  to  comply  with  referral 
(if  given) 

•  SM  satisfaction  with  provider 

•  Ratings  of  provider  patient- 
centered  communication 

Higher  ratings 
of  provider- 
reported 
elicitation  of 

mental  health 

concerns 

Interrupted  time 
series  within  the 
treatment  providers 

•  time  from  the  start 

of  the 

observational 

period 

•  whether  the 
observation  is  pre- 
or  post¬ 
intervention 

•  time  since  the 

intervention 

•  Whether  SM  reported  MH 
symptoms  during  interview  that 
were  not  reported  on  DD  Form 
2900. 

•  Whether  provider  believes  SM 
accurately  reported  all  MH 
symptoms  during  interview. 

•  Whether  provider  believes  SM 
could  benefit  from  further 
evaluation  for  MH  symptoms 
(e.g.,  providers  may  believe  a 
referral  would  be  beneficial  but 
not  have  a  technically  positive 
screen  to  justify). 

Increased 

Mean  comparison 

•  Whether  the 

•  Number  of  health  care 
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number  of 

of  health  care 

observation  is 

encounters  after  the  PDHRA 

health  care 

encounters  before 

from  the 

encounters 

and  after  PDHRA  for 

intervention  or 

after  the 

PDHRA 

SMs  with  trained 
provider  vs.  SMs 
with  untrained 
providers 

comparison  group 

Analytic  models 

Segmented  (or  piecewise)  linear  regression  analyses  will  be  conducted  with 
separate  slopes  of  outcome  for  the  pre-  and  post-intervention  period.  This 
type  of  regression  controls  for  the  baseline  trend  by  testing  the  change  in 
level  and  slope.  The  dependent  variables  will  be  the  outcomes  of  interest 
(provider  concerns,  referrals,  self-efficacy  and  SM  disclosure  and  attitudes) 
and  the  independent  variables  will  be  time  from  the  start  of  the 
observational  period,  whether  the  observation  is  pre-  or  post-  intervention, 
and  time  since  the  intervention.  The  three  levels  of  the  regression  will  be  1) 
slope,  2)  time  within  provider,  and  3)  providers  within  site.  The  analyses  will 
account  for  clustering  of  SMs  within  providers  within  installations. 

In  addition,  potential  moderating  variables  from  the  PDHA  (e.g.,  pre-existing 
mental  health  problems,  combat  exposure),  provider  background  form  (e.g., 
professional  background,  demographics,  self-efficacy  in  patient-centered 
communication),  PDHRA  program  manager  interview  (e.g.,  typical  PDHRA 
processes,  existing  training  programs,  etc.),  and  analysis  of  secondary  data 
(e.g.,  provider  completion  of  the  NDAA  online  training  and  related  test 
scores)  will  be  incorporated  into  the  analytic  models.  These  moderating 
variables  may  affect  the  strength  of  the  relationship  between  the 
independent  and  dependent  variables;  therefore  adding  them  to  the  model 
will  allow  us  to  better  account  for  the  variance  attributable  to  the 
intervention  itself. 

Power  analysis 

A  power  analysis  (Hintze,  2005)  was  conducted  and  it  was  determined  that 
10  providers  across  all  intervention  sites  are  needed  to  detect  medium 
effects  (power  =  80%,  alpha  <  5%  two-tailed).  An  average  cross  wave  (per 
day)  correlation  of  0.9  was  used,  assuming  that  providers  tend  to  behave 
similarly  from  day  to  day.  According  to  Cohen  (1988, 1992),  effect  sizes  of 
about  0.2  are  considered  small;  0.5  are  considered  moderate;  and  0.8  are 
considered  large. 

As  stated  previously,  non-equivalent  comparison  installations  will  also  be 
selected  based  on  criteria  they  have  in  common  with  the  intervention  sites. 
PDHRA  data  will  be  gathered  passively  in  order  to  1)  develop  estimates  for 
installation-  and  provider-level  influences  on  SM  self-reported  problems  and 
referral  patterns,  and  2),  to  inform  the  generalizability  of  installations 
through  a  description  of  the  PDHRA  process.  A  significant  intraclass 
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correlation  (ICC)  would  indicate  that  providers  differ  from  each  other  rather 
than  offering  a  uniform  standard  of  care.  To  see  how  many  providers  are 
needed  to  detect  an  ICC,  we  estimate  power  to  detect  a  small  ICC,  which 
according  to  Raudenbush  (Raudenbush  &  Liu,  2000)  is  ICC  =  0.05.  According 
to  Pass  software  (Hintze,  2005),  if  each  provider  had  20  SMs  and  there  were 
30  providers,  we  would  be  well  powered  (p  =  0.85)  to  detect  a  small  effect. 
Samples  with  fewer  than  30  providers  or  fewer  than  20  SMs  would  have  less 
power  to  detect  a  small  ICC. 
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OVERVIEW 

Due  to  various  factors  described  further  below,  data  collection  totals  resulting  from  the  three  site  visits 
already  completed  do  not  provide  enough  data  for  sufficient  power  to  find  effects  as  per  the  previously 
approved  research  design.  This  amendment  proposes  to  (1)  increase  the  number  of  site  visits  to  four, 
and  (2)  strengthen  the  research  design  to  incorporate  a  naturally  occurring  control  group  of  providers 
who  do  not  participate  in  the  training  workshop.  In  addition,  continued  delays  have  prevented  Purdue 
University  from  receiving  the  data  necessary  to  complete  planned  analyses  to  meet  their  SOW. 

Because  of  the  increased  time  necessary  for  data  collection  we  are  requesting  a  no-cost  extension  of  the 
award  until  SEP  2013.  The  scope  of  work  (SOW)  remains  consistent  with  the  revised  research  plan 
approved  by  MRMC  in  May,  2011.  These  additional  changes  would  simply  allow  Vanderbilt  and  Purdue 
to  complete  the  full  SOW  described  therein.  The  government  sponsor.  Force  Health  Protection  and 
Readiness  (FHP&R),  and  the  coordinating  agency.  Office  of  the  Surgeon  General  of  the  Army  (OTSG)  are 
in  full  support  of  the  proposed  amendment.  The  proposed  amendment  can  be  accomplished  with  no 
additional  cost  to  MRMC. 

JUSTIFICATION 

The  previously  approved  research  design  planned  for  40  Soldier  surveys  to  be  collected  before  and  after 
the  training  workshop  for  each  participating  provider.  This  model  required  a  minimum  of  10-12 
providers  with  adequate  data  collection  to  have  enough  power  to  find  an  effect  of  the  training 
workshop.  Training  workshops  would  be  provided  at  two  to  three  site  visits  with  accompanying  data 
collection.  The  study  has  already  been  implemented  at  three  sites  (Ft  Campbell,  Ft  Stewart,  and  Ft 
Carson),  as  described  in  our  9*'^  quarterly  report.  We  were  not  able  to  collect  sufficient  data  from 
Soldiers  at  any  of  the  site  visits  conducted  so  far. 

Lower  Than  Expected  Soldier  Participation 

Data  has  been  collected  from  19  providers  who  participated  in  the  training  workshops,  but  none  of 
these  had  40  data  points  before  and  after  the  training.  In  fact,  only  1  provider  had  over  35  Soldiers 
complete  a  survey  both  before  and  after  the  workshop.  The  lower  than  expected  data  collection  is  due 
to  at  least  two  factors. 

1.  Estimates  of  the  number  of  Soldiers  going  through  the  installation  for  the  PDHRA  interview  are 
unreliable.  This  number  was  over-estimated  every  day  we  collected  data,  (e.g..  Ft  Campbell 
projected  175  per  day,  but  only  averaged  138  per  day;  Ft  Carson  projected  700  total  PDHRAs,  but 
only  472  were  completed). 

2.  Soldiers  had  to  wait  an  excessive  amount  of  time  to  see  a  provider.  These  delays  negatively 
impacted  their  participation  in  the  study  and  contributed  to  lower  than  expected  response  rates. 

All  Providers  Participated  in  Data  Collection  but  Did  Not  Participate  in  the  Training  Workshop 

Although  it  had  been  anticipated  that  all  providers  present  on  the  day  the  workshop  was  delivered 
would  participate  in  the  workshop,  this  was  not  the  case.  At  all  three  sites,  a  subset  of  providers  was 
chosen  to  participate  in  the  training  workshop  by  site  leadership.  The  reason  given  was  typically  to  allow 
for  continued  PDHRA  flowthrough  during  the  time  of  the  workshop.  To  ease  planning  and  logistics 
involved  with  data  collection  pre-  and  post-PDHRA,  all  sites  chose  to  have  all  providers  participate  in 
data  collection  after  they  had  consented  to  participate.  Thus,  in  addition  to  the  19  providers  who 
participated  in  the  training  workshop,  data  are  available  from  16  providers  who  did  not  participate  in 
the  workshop.  Notably  all  providers  who  were  eligible  consented  to  participate  in  the  study. 
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The  three  tables  below  present  the  actual  data  collection  totals  for  the  Soldier  survey  and  the  Provider  survey  by  provider  at  each  installation. 


Fort  Campbell  Data  Collection  Totals 


Provider  3-qx  surveys  (including  all  completed  by 
providers) 

Soldier  Survey  (from  iPad,  including  only 
surveys  matched  to  a  Provider  3-item 
survey) 

Provider  Name 

Pre-Workshop 

Post-Workshop 

Total 

Pre-Workshop 

Post-Workshop 

Total 

Provider  1* 

31 

27 

58 

16 

15 

31 

Provider  2 

38 

82 

120 

16 

40 

56 

Provider  3* 

57 

67 

124 

39 

40 

79 

Provider  4* 

25 

15 

40 

14 

7 

21 

Provider  5* 

47 

53 

100 

26 

28 

54 

Provider  6 

57 

22 

79 

18 

18 

36 

Provider  7 

23 

57 

80 

13 

23 

36 

Provider  8* 

22 

38 

60 

17 

26 

43 

Provider  9* 

18 

8 

26 

12 

6 

18 

Unmatched  Surveys 

n/a 

n/a 

n/a 

3 

9 

12 

TOTAL  Surveys  at  Fort 
Campbell 

318 

369 

687 

174 

212 

386 

*  Participated  in  training  workshop 
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Fort  Stewart  Data  Collection  Totals 


Provider  3-qx  surveys  (including  all  completed  by 
providers) 

Soldier  Survey  (from  the  iPad,  including  only 
surveys  matched  to  a  Provider  3-item 
survey) 

Provider  Name 

Pre-Workshop 

Post-Workshop 

Total 

Pre-Workshop 

Post-Workshop 

Total 

Provider  1* 

2 

21 

23 

1 

4 

5 

Provider  2 

0 

25 

25 

0 

13 

13 

Provider  3* 

8 

86 

94 

3 

31 

34 

Provider  4 

5 

0 

5 

5 

0 

5 

Provider  5* 

12 

132 

144 

4 

39 

43 

Provider  6* 

7 

88 

95 

5 

32 

37 

Provider  7* 

34 

30 

64 

19 

20 

39 

Provider  8* 

2 

22 

24 

2 

9 

11 

Provider  9* 

33 

25 

58 

10 

8 

18 

Provider  10* 

3 

27 

30 

1 

15 

16 

Provider  11* 

2 

69 

71 

2 

24 

26 

Provider  12 

0 

1 

1 

0 

0 

0 

UNMATCHED  SURVEYS 

n/a 

n/a 

n/a 

3 

13 

16 

TOTAL  Surveys  at  Fort 
Stewart 

108 

505 

634 

55 

208 

263 

*  Participated  in  training  workshop.  Yellow  highlight  indicates  less  than  5  surveys  collected. 
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Fort  Carson  Data  Collection  Totals 


Provider  3-qx  surveys  (including  all  completed  by 

Soldier  Survey  (from  the  iPad,  including  only 

providers) 

surveys  matched  to  a  Provider  3-item  survey) 

Provider  Name 

Pre-Workshop 

Post-Workshop 

Total 

Pre-Workshop 

Post-Workshop 

Total 

Provider  1 

1 

4 

5 

0 

0 

0 

Provider  2* 

62 

25 

87 

19 

4 

23 

Provider  3 

4 

8 

12 

3 

2 

5 

Provider  4 

29 

12 

41 

13 

5 

18 

Provider  5* 

57 

26 

83 

11 

4 

15 

Provider  6* 

35 

18 

53 

10 

8 

18 

Provider  7 

6 

25 

31 

1 

3 

4 

Provider  8 

2 

13 

15 

1 

6 

7 

Provider  9 

4 

4 

8 

1 

0 

1 

Provider  10 

1 

5 

6 

0 

1 

1 

Provider  11 

2 

16 

18 

1 

4 

5 

Provider  12 

0 

6 

6 

0 

4 

4 

Provider  13* 

53 

26 

79 

23 

11 

34 

Provider  14 

7 

18 

25 

2 

5 

7 

UNMATCHED  SURVEYS 

n/a 

n/a 

n/a 

5 

5 

10 

TOTAL  Surveys  at  Fort  Carson 

263 

206 

469 

90 

62 

152 

*  Participated  in  training  workshop.  Yellow  highlight  indicates  less  than  5  surveys  collected. 
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Although  Soldier  participation  was  expected  to  average  around  50%,  in  actuality  it  ranged  from  12-32%  at  all 
sites.  The  table  below  summarizes  the  total  data  collection  for  all  three  sites.  The  average  number  of 
matched  surveys  (i.e.,  presence  of  both  a  Soldier  survey  and  a  provider  survey)  collected  per  provider  per  day 
ranged  from  3-6. 


Summary  of  Data  Collection  To  Date  by  Installation 


Ft  Campbell 

Ft  Stewart 

Ft  Carson 

days  of  data  collection 

7 

7 

4 

number  of  providers* 

9 

12 

14 

estimated  flowthrough 

1225 (175  per  day) 

2170 (310  per  day) 

568  (142  per  day) 

actual  flowthrough 

966  (138  per  day) 

804(115  per  day) 

472 (118  per  day) 

provider  3-qx  surveys 
collected 

687 

634 

469 

matched  SM  surveys  collected 

386 

263 

152 

%  SM  participation  based  on 
estimated  flowthrough 

32 

12 

27 

Average  #  matched  SM 
surveys  per  day  per  provider 

6 

3 

3 

*who  completed  at  least  one  3-qx  survey;  fewer  providers  have  adequate  pre/post  data 
and  particiapted  in  the  workshop 

Continued  delays  in  receiving  VA  data 

As  of  27  FEB  2012,  Purdue  has  still  not  received  the  VA  data  necessary  to  complete  their  proposed  analysis. 
The  key  innovative  aspect  of  Purdue's  subcontract  is  the  merging  of  PDHA/PDHRA  data,  health  care 
encounter  data  from  AFHSC,  and  health  care  encounter  data  from  the  National  VA  Data  Repository  to 
examine  risk  factors  associated  with  the  development  of  PTSD  in  post-deployment  OIF/OEF  Service 
members.  To  date,  Purdue  has  received  the  PDFIA/PDFIRA  and  AFFISC  health  care  encounter  data  from 
Vanderbilt,  but  has  yet  to  receive  the  national  VA  data.  Researchers'  access  to  national  VA  data  with  real 
SSNs  requires  submission  of  a  request  packet  to  National  Data  Systems  (NDS)  with  a  completed  form  9957 
and  local  approvals.  Purdue's  VA  collaborator  submitted  all  necessary  forms,  approvals,  and  signatures  early 
in  2011  and  received  permission  to  access  the  database  in  August,  2011.  At  that  time,  there  was  an 
apparent  change  in  the  approvals  procedure  in  which  data  requests  were  required  to  be  routed  through 
VFIA  Privacy,  VHA  Security,  and  the  VHA  Office  of  Research  and  Development  (ORD).  The  project  has 
received  approvals  from  Privacy  and  Security  in  October,  2011,  but  is  still  waiting  on  ORD  approval. 
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PROPOSED  STUDY  CHANGES  FOR  APPROVAL  BY  MRMC 

Aim  2  Design  and  Methodology 

Currently,  we  have  a  single-group  design  with  pre-  and  post-workshop  measures  and  time-series  data 
(secondary  data,  e.g.,  PDHRA,  health  care  encounters).  This  includes  only  providers  who  participate  in  the 
workshop  and  emphasizes  the  number  of  Soldiers  over  the  number  of  providers.  The  proposed  quasi- 
experimental  design  uses  all  collected  data  and  allows  us  to  have  fewer  data  points  per  provider  (i.e., 
minimum  of  5  matched  provider  3-question  surveys  and  Soldier  surveys). 

Our  proposed  quasi-experiment  includes  two  groups,  experimental  (participated  in  the  workshop)  and 
control  (did  not  participate  in  the  workshop).  Based  on  current  data  collected  we  have  11  providers  in  the 
experimental  group  with  adequate  data  (5  or  more  matched  Soldier  and  provider  surveys)  and  4  providers 
in  the  control  group  with  adequate  data.  The  benefits  of  this  design  are  that  it  controls  for  major  threats 
to  validity  such  as  unknown  effects  of  time  or  other  events,  and  it  increases  statistical  power  by  increasing 
the  total  number  of  providers,  which  allows  us  to  collect  fewer  data  points  (i.e..  Soldier-specific  data)  from 
each  provider.  Limitations  of  the  design  are  that  providers  are  not  randomized  to  groups  (thus,  the  quasi¬ 
experiment),  and  that  the  two  groups  are  currently  unbalanced  (11  and  4),  which  decreases  power. 

Power  Analysis 

The  chart  below  shows  the  power  analysis  we  used  to  determine  the  average  number  of  providers  in  each 
group  (control  and  experimental)  we  would  need  to  show  medium  effects  of  the  training.  In  previously 
collected  PDHRA  data,  the  intraclass  correlation  coefficient  (ICC)  is  0.33  for  the  effect  of  provider  variance 
on  whether  any  referral  was  made,  and  0.56  for  the  effect  of  provider  variance  on  whether  any  behavioral 
health  concerns  were  documented  on  the  PDHRA.  Therefore,  we  estimate  that  provider  variance  is  fairly 
robust  necessitating  an  anticipated  cross-wave  r  (left-hand  column)  between  0.30  and  0.55.  We  then 
targeted  the  columns  for  medium  size  effects  (ranging  from  0.50  to  0.70). 
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To  create  equal-sized  groups  with  adequate  power  to  detect  intermediate  effect  sizes  we  need  15 
additional  providers  -  4  in  the  experimental  group  and  11  in  the  control  group.  Each  provider  needs  to 
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complete  15-20  PDHRA  interviews  before  and  after  the  training  (or  30-40  interview  total  for  the  control 
group),  and  30%  of  the  Soldiers  they  interview  must  agree  to  participate  in  the  Soldier  survey.  This  is 
consistent  with  participation  at  prior  site  visits. 

We  are  targeting  Ft  Bliss,  TX,  for  the  additional  site  visit.  This  selection  is  based  on  the  presence  of  a  large 
number  of  potential  participants  (up  to  20  providers),  and  a  large  number  of  Soldiers  expected  to 
complete  the  PDHRA  process  in  MAR  2012  (200-250  Soldiers  per  day).  The  PDHRA  administrators  at  Ft 
Bliss  are  supportive,  and  OTSG  has  issued  a  tasking  order  and  is  coordinating  a  letter  of  support  from  the 
installation  commander.  If  approved,  we  expect  the  site  visit  to  occur  in  MAR  2012.  We  expect  the  site 
visit  to  last  about  10  days  to  ensure  adequate  data  collection  if  flow  through  or  Soldier  participation  is 
lower  than  expected. 

Expected  Data  Requests 

Army 

The  data  request  to  the  Army  will  be  an  amendment  to  the  existing  data  request  to  include  data  for  the 
intervention  (i.e.,  for  Soldiers  who  completed  PDHRAs  within  6  months  before  to  4  weeks  after  the 
intervention).  Due  to  the  length  of  time  between  completed  site  visits  and  the  visit  planned  for  MAR 
2012,  we  will  request  these  data  in  two  installments.  The  first  request  will  include  data  from  20  APR  2011 
to  20  JAN  2012,  and  the  second  request  will  include  data  from  21  JAN  2012  thru  approximately  08  JUN 
2012.  This  amendment  has  been  submitted  as  of  25  JAN  2012. 

Armed  Forces  Health  Surveillance  Center  (AFHSC) 

The  data  use  agreement  for  these  data  has  been  approved.  Like  the  Army  data  request,  the  AFHSC  data 
request  will  occur  in  two  installments,  including  the  same  date  ranges  as  for  the  Army  request. 

Analytic  Models 

All  hypotheses  and  research  questions  will  remain  the  same,  with  the  addition  of  analytic  models  that 
provide  comparisons  of  the  experimental  and  control  groups.  For  these  analyses,  segmented  (or 
piecewise)  linear  regression  analyses  will  be  conducted  with  separate  slopes  of  outcome  for  the  pre-  and 
post-intervention  period.  This  type  of  regression  controls  for  the  baseline  trend  by  testing  the  change  in 
level  and  slope.  The  dependent  variables  will  be  the  outcomes  of  interest  (provider  concerns,  referrals, 
self-efficacy  and  Soldier  disclosure  and  attitudes)  and  the  independent  variables  will  be  time  from  the 
start  of  the  observational  period,  whether  the  observation  is  pre-  or  post-  intervention,  and  time  since  the 
intervention.  The  three  levels  of  the  regression  will  be  1)  slope,  2)  time  within  provider,  and  3)  providers 
within  site.  The  analyses  will  account  for  clustering  of  Soldiers  within  providers  and  within  installations. 

In  addition,  potential  moderating  variables  from  the  PDHA  (e.g.,  pre-existing  mental  health  problems, 
combat  exposure),  provider  background  form  (e.g.,  professional  background,  demographics,  self-efficacy 
in  patient-centered  communication),  PDHRA  program  manager  interview  (e.g.,  typical  PDHRA  processes, 
existing  training  programs,  etc.),  and  analysis  of  secondary  data  (e.g.,  provider  completion  of  the  NDAA 
online  training  and  related  test  scores)  will  be  incorporated  into  the  analytic  models.  These  moderating 
variables  may  affect  the  strength  of  the  relationship  between  the  independent  and  dependent  variables; 
therefore  adding  them  to  the  model  will  allow  us  to  better  account  for  the  variance  attributable  to  the 
intervention  itself. 

IMPACT  OF  NO-COST  EXTENSION  YEAR 

Due  to  the  need  for  an  additional  site  visit  and  the  delay  in  Purdue  receiving  the  VA  data,  we  propose  a 
no-cost  extension  year  until  SEP  2013.  The  intent  of  this  extension  is  to  allow  completion  of  the  SOW 
outlined  in  the  revised  research  plan  approved  by  MRMC  in  May  2011. 
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With  the  additional  site  visit,  all  data  will  not  be  received  by  Vanderbilt  until  June  2012  at  earliest.  It  is 
more  likely  that  final  data  will  not  be  received  by  Vanderbilt  until  July  or  August  2012  given  the  need  for 
multiple  data  requests  and  coordination  of  data  retrieval  as  described  above.  In  addition,  access  to  the 
AFHSC  data  will  expire  at  the  end  of  the  contract  period  in  the  fall  of  2012.  The  additional  year  would 
allow  time  for  Vanderbilt  to  receive  data  related  to  all  four  study  sites,  clean  and  merge  datasets, 
complete  analyses  with  data  from  all  four  sites,  write  manuscripts  and  a  final  report. 

While  waiting  for  access  to  the  VA  system,  Purdue  has  been  analyzing  the  PDHA/PDHRA  and  AFHSC  health 
care  encounter  data.  Purdue  is  currently  writing  2  manuscripts  that  address  the  original  research  question 
with  the  limited  data  they  have.  However,  because  they  have  not  received  the  VA  data  yet,  Purdue  will  be 
unable  to  complete  the  project  as  specified  in  their  statement  of  work  by  the  end  of  the  current  project 
period  unless  they  have  additional  time.  One  additional  year  would  allow  Purdue  time  to  receive  the  VA 
data,  clean,  code  and  merge  it,  complete  analyses,  and  write  manuscripts  and  a  final  report. 
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Proposal  Title:  "Improving  Deployment-Related  Primary  Care  Provider  Assessments  of  PTSD  and  Mental 
Health  Conditions." 

Period  of  Performance:  30  September  2009  -  31  October  2013  (Research  ends  29  September  2013). 
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A.  This  award  is  made  under  the  authority  of  31  U.S.C.  6305  and  10  U.S.C.  2358.  The  recipient's 
statement  of  work  on  pages  6  through  18  of  the  proposal  dated  25  March  2009  and  the  revised  budget 
dated  21  September  2009  on  pages  1  through  23  are  incorporated  herein  by  reference.  The  Catalog  of 
Federal  Domestic  Assistance  Number  relative  to  this  award  is  CFDA  12.420. 

GOVERNMENT  INTERACTION  (DEC  2008){USAMRAA) 

The  active  participants  in  this  award  are  the  U.S.  Army  Medical  Research  and  Materiel  Command 
(USAMRMC)  and  its  laboratories  identified  herein  through  the  U.S.  Army  Medical  Research  Acquisition 
Activity  (USAMRAA).  The  following  USAMRMC  laboratory  will  be  the  focus  of  cooperative  research 
conducted  under  this  agreement: 

o  Ft.  Campbell,  Ky. 

o  Force  Flealth  Protection  &  Readiness  (FFIR&P). 

B.  ACCEPTANCE  OF  AWARD.  The  recipient  is  not  required  to  countersign  this  assistance  award.  In 
case  of  disagreement,  the  recipient  shall  notify  the  Grants  Officer  and  not  assess  the  award  any  costs 
until  such  disagreement(s)  is  resolved. 

C.  MAXIMUM  OBLIGATION  (SEP  2006)  (USAMRAA) 

The  maximum  obligation  for  support  of  the  project  will  not  exceed  the  amount  specified  in  the  award, 
as  amended.  USAMRAA  does  not  amend  assistance  agreements  to  provide  additional  funds  for  such 
purposes  as  reimbursement  for  unrecovered  indirect  costs  resulting  from  the  establishment  of  final 
negotiated  rates  or  for  increases  in  salaries,  fringe  benefits  and  other  costs. 

D.  TERMS  AND  CONDITIONS:  The  recipient  agrees  to  the  General  Terms  and  Conditions  of  the  Federal 
Demonstration  Partnership,  Phase  V,  dated  July  1,  2008  and  Department  of  Army  -  Agency  Specific 
Requirements.  Modifications  to  the  General  Terms  and  Conditions  dated  July  1,  2008  are  modified  as 
indicated  below. 

1.  PATENTS  AND  INVENTIONS  (DEC  2001)  (USAMRAA) 

a.  The  recipient  shall  use  the  Interagency  Edison  through  the  National  Institutes  of  Flealth 
Commons  (http://www.iedison.gov/)  for  filing  of  Patent  Application  and  Invention  Disclosure.  Negative 
reports  are  required  and  shall  be  submitted  on  a  DD  Form  882  to  the  Grants  Officer.  (DD  Form  882  can 
be  located  on  web  site  http://www.usamraa.army.mil ). 

b.  Invention  reports  are  due  annually  and  at  the  end  of  the  period  of  the  award.  Annual  reports 
are  due  30  days  after  the  anniversary  date  of  the  award  and  final  reports  are  due  30  days  after  the 
expiration  of  the  award.  The  award  will  NOT  be  closed  out  until  all  invention  reporting  requirements  are 
met. 

2.  TECHNICAL  REPORTING  REQUIREMENTS  (DEC  2008)  (USAMRAA) 
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Format  Requirements  for  Annual/Final  Reports 

a.  Annual  reports  must  provide  a  complete  summary  of  the  research  accomplishments  to  date 
with  respect  to  the  approved  Statement  of  Work.  Journal  articles  can  be  substituted  for  detailed 
descriptions  of  specific  aspects  of  the  research,  but  the  original  articles  must  be  attached  to  the  report 
as  an  appendix  and  appropriately  referenced  in  the  text.  The  importance  of  the  report  to  decisions 
relating  to  continued  support  of  the  research  cannot  be  over-emphasized.  An  annual  report  shall  be 
submitted  within  30  calendar  days  of  the  anniversary  date  of  the  award  for  the  preceding  12  month 
period.  If  the  award  period  of  performance  is  extended  by  the  Grants  Officer,  then  an  annual  report 
must  still  be  submitted  within  30  days  of  the  anniversary  date  of  the  award.  A  final  report  will  be  due 
upon  completion  of  the  extended  performance  date  that  describes  the  entire  research  effort. 

b.  A  final  report  summarizing  the  entire  research  effort,  citing  data  in  the  annual  reports  and 
appended  publications  shall  be  submitted  at  the  end  of  the  award  performance  period.  The  final  report 
will  provide  a  complete  reporting  of  the  research  findings.  Journal  publications  can  be  substituted  for 
detailed  descriptions  of  specific  aspects  of  the  research,  but  an  original  copy  of  each  publication  must  be 
attached  as  an  appendix  and  appropriately  referenced  in  the  text.  All  final  reports  must  include  a 
bibliography  of  all  publications  and  meeting  abstracts  and  a  list  of  personnel  (not  salaries)  receiving  pay 
from  the  research  effort. 

Although  there  is  no  page  limitation  for  the  reports,  each  report  shall  be  of  sufficient  length  to 
provide  a  thorough  description  of  the  accomplishments  with  respect  to  the  approved  Statement  of 
Work.  Submission  of  the  report  in  electronic  format  (PDF  or  Word  file  only),  shall  be  submitted  to 
https://ers.amedd.army.mil. 

All  reports  shall  have  the  following  elements  in  this  order 

FRONT  COVER:  Sample  front  cover  provided  at  https://mrmc.amedd.army.mil/rrpindex.asp.  The 
Accession  Document  (AD)  Number  should  remain  blank. 

STANDARD  FORM  298:  Sample  SF  298  provided  at  https://mrmc.amedd.army.mil/rrpindex.asp.  The 
abstract  in  Block  13  must  state  the  purpose,  scope,  major  findings  and  be  an  up-to-date  report  of  the 
progress  in  terms  of  results  and  significance.  Subject  terms  are  keywords  that  may  have  previously 
assigned  to  the  proposal  abstract  or  are  keywords  that  may  be  significant  to  the  research.  The  number 
of  pages  shall  include  all  pages  that  have  printed  data  (including  the  front  cover,  SF  298,  table  of 
contents,  and  all  appendices).  Please  count  pages  carefully  to  ensure  legibility  and  that  there  are  no 
missing  pages  as  this  delays  processing  of  reports.  Page  numbers  should  be  typed:  please  do  not  hand 
number  pages. 

TABLE  OF  CONTENTS:  Sample  table  of  contents  provided  at 
https://mrmc.amedd.armv.mil/rrpindex.asp. 

INTRODUCTION:  Narrative  that  briefly  (one  paragraph)  describes  the  subject,  purpose  and  scope  of 
the  research. 

BODY:  This  section  of  the  report  shall  describe  the  research  accomplishments  associated  with  each 
task  outlined  in  the  approved  Statement  of  Work.  Data  presentation  shall  be  comprehensive  in 
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providing  a  complete  record  of  the  research  findings  for  the  period  of  the  report.  Provide  data 
explaining  the  relationship  of  the  most  recent  findings  with  that  of  previously  reported  findings. 
Appended  publications  and/or  presentations  may  be  substituted  for  detailed  descriptions  of 
methodology  but  must  be  referenced  in  the  body  of  the  report.  If  applicable,  for  each  task  outlined  in 
the  Statement  of  Work,  reference  appended  publications  and/or  presentations  for  details  of  result 
findings  and  tables  and/or  figures.  The  report  shall  include  negative  as  well  as  positive  findings.  Include 
problems  in  accomplishing  any  of  the  tasks.  Statistical  tests  of  significance  shall  be  applied  to  all  data 
whenever  possible.  Figures  and  graphs  referenced  in  the  text  may  be  embedded  in  the  text  or 
appended.  Figures  and  graphs  can  also  be  referenced  in  the  text  and  appended  to  a  publication. 
Recommended  changes  or  future  work  to  better  address  the  research  topic  may  also  be  included, 
although  changes  to  the  original  Statement  of  Work  must  be  approved  by  the  Army  Grants  Officer's 
Representative.  This  approval  must  be  obtained  prior  to  initiating  any  change  to  the  original  Statement 
of  Work. 

KEY  RESEARCFI  ACCOMPLISFIMENTS:  Bulleted  list  of  key  research  accomplishments  emanating  from 
this  research. 

REPORTABLE  OUTCOMES:  Provide  a  list  of  reportable  outcomes  that  have  resulted  from  this  research 
to  include: 

manuscripts,  abstracts,  presentations;  patents 
and  licenses  applied  for  and/or  issued;  degrees 
obtained  that  are  supported  by  this  award; 
development  of  cell  lines,  tissue  or  serum 
repositories;  informatics  such  as  databases  and 
animal  models,  etc.;  funding  applied  for  based 
on  work  supported  by  this  award;  employment 
or  research  opportunities  applied  for  and/or 
received  based  on  experience/training  supported 
by  this  award. 

CONCLUSION:  Summarize  the  results  to  include  the  importance  and/or  implications  of  the  completed 
research  and  when  necessary,  recommend  changes  on  future  work  to  better  address  the  problem.  A 
"so  what  section"  which  evaluates  the  knowledge  as  a  scientific  or  medical  product  shall  also  be 
included  in  the  conclusion  of  the  report. 

REFERENCES:  List  all  references  pertinent  to  the  report  using  a  standard  journal  format  (i.e.  format 
used  in  Science,  Military  Medicine,  etc.). 

APPENDICES:  Attach  all  appendices  that  contain  information  that  supplements,  clarifies  or  supports 
the  text.  Examples  include  original  copies  of  journal  articles,  reprints  of  manuscripts  and  abstracts,  a 
curriculum  vitae,  patent  applications,  study  questionnaires,  and  surveys,  etc. 

Pages  shall  be  consecutively  numbered  throughout  the  report.  DO  NOT  RENUMBER  PAGES  IN  TFIE 
APPENDICES. 

Mark  all  pages  of  the  report  which  contain  proprietary  or  unpublished  data  that  should  be  protected 
by  the  U.S.  Government.  REPORTS  NOT  PROPERLY  MARKED  FOR  LIMITATION  WILL  BE  DISTRIBUTED  AS 
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APPROVED  FOR  PUBLIC  RELEASE.  It  is  the  responsibility  of  the  Principal  Investigator  to  advise  the  U.S. 
Army  Medical  Research  and  Materiel  Command  when  restricted  limitation  assigned  to  a  document  can 
be  downgraded  to  Approved  for  Public  Release.  DO  NOT  USE  THE  WORD  "CONFIDENTIAL"  WHEN 
MARKING  DOCUMENTS. 

3.  PAYMENTS 


ADVANCE  PAYMENTS  AND  FULL  FUNDING  (DEC  2008)  (USAMRAA) 

a.  Payments.  Advance  payments  will  be  made  to  the  recipient.  Questions  relative  to  payment 
issues  involving  Defense  Finance  and  Accounting  Service  shall  be  directed  to  Craig  E.  Anderson  @  301- 
619-2702  or  craig.e.anderson@amedd.army.mil. 

b.  Electronic  Funds  Transfer.  All  advance  payments  to  the  recipient  will  be  made  by  electronic 
funds  transfer  (EFT).  The  recipient  shall  contact  the  Defense  Finance  and  Accounting  System  (DFAS) 
named  on  the  face  page  of  this  award  to  make  arrangements  for  EFT.  Failure  to  do  so  may  result  in 
nonpayment. 

c.  If  the  recipient  fails  to  perform,  the  Grants  Officer  shall  notify  DFAS  in  writing  to 
withhold  payments. 


d.  Advance  Payment  Schedule 
Year  One:  $690,431.00 


Amount 


$172,607.75 

$172,607.75 

$172,607.75 

$172,607.75 

Year  Two:  $681,502.00 


Amount 


$170,375.50 

$170,375.50 

$170,375.50 

$170,375.50 


On  or  About 


Upon  execution  of  this  award 
30  December  2009 
30  March  2010 
30  June  2010 


On  or  About 

30  September  2010 
30  December  2010 
30  March  2011 
30  June  2011 


e.  Financial  Reporting  Requirements:  The  recipient  shall  submit  on  a  quarterly  basis  a  Standard 
Form  272,  Federal  Cash  Transactions  Report  (form  available  on  web  site  http://www.usamraa.army.mil). 
Each  report  shall  be  completed  in  U.S.  dollars  and  submitted  to  the  U.S.  Army  Medical  Research 
Acquisition  Activity,  ATTN:  MCMR-AAA-W,  820  Chandler  Street,  Fort  Detrick  MD  21702-5014  in 
accordance  with  the  following  schedule: 
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Period  Covered  Due  Date 


Jan  -  Mar 
Apr  -  Jun 
Jul  -  Sep 
Oct  -  Dec 


15  Apr 
15  Jul 
15  Oct 
15  Jan 


f.  Interest  Bearing  Account.  Unless  exempted  by  applicable  Treasury-State  agreements  in 
accordance  with  the  Cash  Management  Improvement  Act  (CMIA)  (31  U.S.C.  3335),  the  recipient  shall 
deposit  all  advance  payments  in  an  interest  bearing  account.  Interest  over  the  amount  of  $250  per  year 
shall  be  remitted  annually  to  the  Department  of  Health  and  Human  Services,  Payment  Management 
System,  P.O.  Box  6021,  Rockville,  MD  20852.  A  copy  of  the  transmittal  letter  stating  the  amount  of 
interest  remitted  shall  be  sent  to  the  U.S.  Army  Medical  Research  Acquisition  Activity,  ATTN:  MCMR- 
AAA-W,  820  Chandler  Street,  Fort  Detrick,  MD  21702-5014. 

4.  PROHIBITION  OF  USE  OF  HUMAN  RESEARCH  (JAN  2007)  (USAMRAA) 

**  PROHIBITION  -  READ  FURTHER  FOR  DETAILS  ** 

Research  under  this  award  involving  the  use  of  human  subjects,  to  include  the  use  of  human  anatomical 
substances  and/or  human  data,  may  not  begin  until  the  U.S.  Army  Medical  Research  and  Materiel 
Command's  Office  of  Research  Protections,  Human  Research  Protections  Office  (HRPO)  approves  the 
protocol.  Written  approval  to  begin  research  or  subcontract  for  the  use  of  human  subjects  under  the 
applicable  protocol  proposed  for  this  award  will  be  issued  from  the  US  Army  Medical  Research  and 
Materiel  Command,  HRPO,  under  separate  letter  to  the  recipient.  A  copy  of  this  approval  will  be 
provided  to  the  US  Army  Medical  Research  Acquisition  Activity  for  the  official  file.  Non-compliance  with 
any  provision  of  this  clause  may  result  in  withholding  of  funds  and  or  the  termination  of  the  award. 

5.  PROHIBITION  OF  USE  OF  LABORATORY  ANIMALS  (JAN  2007)  (USAMRAA) 

**  PROHIBITION  -  READ  FURTHER  FOR  DETAILS  ** 

Notwithstanding  any  other  provisions  contained  in  this  award  or  incorporated  by  reference  herein,  the 
recipient  is  expressly  forbidden  to  use  or  subcontract  for  the  use  of  laboratory  animals  in  any  manner 
whatsoever  without  the  express  written  approval  of  the  US  Army  Medical  Research  and  Materiel 
Command,  Animal  Care  and  Use  Office  (ACURO).  The  recipient  will  receive  written  approval  to  begin 
research  under  the  applicable  protocol  proposed  for  this  award  from  the  US  Army  Medical  Research  and 
Materiel  Command,  ACURO,  under  separate  letter.  A  copy  of  this  approval  will  be  provided  to  the  US 
Army  Medical  Research  and  Acquisition  Activity  for  the  official  file.  Non-compliance  with  any  provision 
of  this  clause  may  result  in  the  termination  of  the  award. 

6.  PROHIBITION  OF  USE  OF  HUMAN  CADAVERS  (JAN  2007)  (USAMRAA) 

**  PROHIBITION  -  READ  FURTHER  FOR  DETAILS** 

Research  under  this  award  using  human  cadavers  may  not  begin  until  the  U.S.  Army  Medical  Research 
and  Materiel  Command's  Office  of  Research  Protections,  Human  Research  Protections  Office  (HRPO) 
approves  the  protocol.  Written  approval  to  begin  research  or  subcontract  for  the  use  of  human  cadavers 
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under  the  applicable  protocol  proposed  for  this  award  will  be  issued  from  the  US  Army  Medical 
Research  and  Materiel  Command,  HRPO,  under  separate  letter  to  the  recipient.  A  copy  of  this  approval 
will  be  provided  to  the  US  Army  Medical  Research  Acquisition  Activity  for  the  official  file.  Non- 
compliance  with  any  provision  of  this  clause  may  result  in  withholding  of  funds  and  or  the  termination 
of  the  award. 

7.  SUPPORTING  INFORMATION  (APR  2008)  (USAMRAA) 

Information  such  as  subawards,  consultant  agreements,  vendor  quotes,  and  personnel  work  agreements 
may  be  required  in  order  to  support  proposed  costs  or  to  determine  the  employment  status  of 
personnel  under  the  assistance  agreement.  The  Government's  receipt  of  this  information  does  not 
constitute  approval  or  acceptance  of  any  term  or  condition  included  therein.  The  terms  and  conditions 
of  the  assistance  agreement  take  precedence  over  any  term  or  condition  included  in  supporting 
information. 

8.  TRAFFICKING  VICTIMS  PROTECTION  ACT  (May  2008)  (USAMRAA) 

Trafficking  in  persons. 

a.  Provisions  applicable  to  a  recipient  that  is  a  private  entity. 

1.  You  as  the  recipient,  your  employees,  subrecipients  under  this  award,  and  subrecipients' 
employees  may  not- 

i.  Engage  in  severe  forms  of  trafficking  in  persons  during  the  period  of  time  that  the  award 
is 

in  effect; 

ii.  Procure  a  commercial  sex  act  during  the  period  of  time  that  award  is  in  effect;  or 

iii.  Use  forced  labor  in  the  performance  of  the  award  or  subawards  under  the  award. 

2.  We  as  the  Federal  awarding  agency  may  unilaterally  terminate  this  award,  without  penalty,  if 
you  or  a  subrecipient  that  is  a  private  entity- 

i.  Is  determined  to  have  violated  a  prohibition  in  paragraph  a.l  of  this  award  term;  or 

ii.  Has  an  employee  who  is  determined  by  the  agency  official  authorized  to  terminate  the 
award  to  have  violated  a  prohibition  in  paragraph  a.l  of  this  award  term  through  conduct  that  is  either- 

A.  Associated  with  performance  under  this  award;  or 

B.  Imputed  to  you  or  the  subrecipient  using  the  standards  and  due  process  for  imputing 
the  conduct  of  an  individual  to  an  organization  that  are  provided  in  2  CFR  180,  "0MB  Guidelines  to 
Agencies  on  Government  wide  Debarment  and  Suspension  (Nonprocurement),"  as  implemented  by  our 
agency  at  2  CFR  part  1125. 

b.  Provision  applicable  to  a  recipient  other  than  a  private  entity.  We  as  the  Federal  awarding 
agency  may  unilaterally  terminate  this  award,  without  penalty,  if  a  subrecipient  that  is  a  private  entity- 

I.  Is  determined  to  have  violated  an  applicable  prohibition  in  paragraph  a.l  of  this  award  term; 
or 
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2.  Has  an  employee  who  is  determined  by  the  agency  official  authorized  to  terminate  the  award 
to  have  violated  an  applicable  prohibition  in  paragraph  a.l  of  this  award  term  through  conduct  that  is 
either-- 


i.  Associated  with  performance  under  this  award; 

ii.  Imputed  to  the  subrecipent  using  the  standards  and  due  process  for  imputing  the 
conduct  of  an  individual  to  an  organization  that  are  provided  in  2  CFR  part  180,  "0MB  Guidelines  to 
Agencies  on  Governmentwide  Debarment  and  Suspension  (Nonprocurement),"  as  implemented  by  our 
agency  at  2  CFR  part  1125. 

c.  Provision  applicable  to  any  recipient. 

1.  You  must  inform  us  immediately  of  any  information  you  receive  from  any  source  alleging  a 
violation  of  a  prohibition  in  paragraph  a.l  of  the  award  term. 

2.  Our  right  to  terminate  unilaterally  that  is  described  in  paragraph  a. 2.  or  b.  of  this  section: 

i.  Implements  section  106(g)  of  the  Trafficking  Victims  Protection  Act  of  2000  (TVPA),  as 
amended  (22  U.S.C.  7104(g)),  and 

ii.  Is  in  addition  to  all  other  remedies  for  noncompliance  that  are  available  to  us  under  this 

award. 


3.  You  must  include  the  requirements  of  paragraph  a.l  of  this  award  term  in  any  subaward  you 
make  to  a  private  entity. 

d.  Definitions.  For  the  purpose  of  this  award  term: 

1.  "Employee"  means  either: 

i.  An  individual  employed  by  you  or  a  subrecipient  who  is  engaged  in  the  performance  of 
the  project  or  program  under  this  award;  or 

ii.  Another  person  engaged  in  the  performance  of  the  project  or  program  under  this  award 
and  not  compensated  by  you  including,  but  not  limited  to,  a  volunteer  or  individual  whose  services  are 
contributed  by  a  third  party  as  an  in-kind  contribution  toward  cost  sharing  or  matching  requirements. 

2.  "Forced  labor"  means  labor  obtained  by  any  of  the  following  methods:  the  recruitment, 
harboring,  transportation,  provision,  or  obtaining  of  a  person  for  labor  or  services,  through  the  use  of 
force,  fraud,  or  coercion  for  the  purpose  of  subjection  to  involuntary  servitude,  peonage,  debt  bondage, 
or  slavery. 

3.  "Private  entity": 

i.  Means  any  entity  other  than  a  State,  local  government,  Indian  Tribe,  or  foreign  public 
entity,  as  those  terms  are  defined  in  2  CFR  175.25. 

ii.  Includes: 
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A.  A  nonprofit  organization,  including  any  nonprofit  institution  of  higher  education, 
hospital,  or  tribal  organization  other  than  one  included  in  the  definition  if  Indian  Tribe  at  2  CFR 
175.25(b). 

B.  A  for-profit  organization. 

4.  "Severe  forms  of  trafficking  in  persons,"  "commercial  sex  act,"  and  "coercion"  have  the 
meanings  given  at  section  103  of  the  TVPA,  as  amended  (22  U.S.C.  7102). 

9.  OPTION  TO  EXTEND  THE  TERM  OF  THE  ASSISTANCE  AGREEMENT  (MAR  2008)  (USAMRAA) 

a.  The  Government  may  extend  the  term  of  this  assistance  agreement  by  issuance  of  a 
modification  that  provides  additional  funding  for  continued  performance  of  the  research  effort.  The 
modification  will  be  issued  prior  to  the  expiration  date  of  the  assistance  agreement.  Inclusion  of  this 
term  does  not  commit  the  Government  to  an  extension. 

b.  Exercise  of  this  option  is  contingent  upon  receipt  of  appropriated  funds  and  acceptable 
performance  by  the  recipient. 

c.  If  the  Government  exercises  this  option,  the  extended  assistance  agreement  shall  be 
considered  to  include  this  option  term. 

10.  TRAVEL-GOVERNMENT/MILITARY  PERSONNEL  (MARCH  2008)  (USAMRAA) 

Travel  costs  associated  with  the  Government/Military  personnel  providing  research  effort  on  this  award 
are  authorized  in  accordance  with  Title  31  U.S.C.  Section  1353. 


The  following  have  been  deleted: 

USAMRAA-XXXX-  FDP  AGENCY  SPECIFIC  TERMS  AUG  2009 

0002 


SECTION  00800  -  SPECIAL  CONTRACT  REQUIREMENTS 


The  following  have  been  added  by  full  text: 
SPECIAL  CONTRACT  REQUIREMENTS 
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ADDITIONAL  TECHNICAL  REPORTING  REQUIREMENTS: 

PROGRAMMATIC  LINE  REVIEW  (PLR) 


a.  The  reporting  requirements  for  Military  Operational  Medicine  Research  Program  (MOMRP) 
include  quarterly,  annual  and  final  reports  and  the  Principal  Investigator's  (Pi's)  participation  in  at  least 
one  programmatic  line  review  (PLR)  for  this  project  each  year  of  the  project's  period-of-performance. 

b.  The  PI  shall  prepare  for  and  participate  in  at  least  one  PLR  for  this  project  for  each  year  of  the 
project's  term,  at  the  Grants  Officer's  Representative's  (GOR's)  request.  The  invitation  and  format  for 
the  programmatic  review  will  be  provided  by  MOMRP  at  least  90  days  prior  to  the  meeting.  The 
meetings  will  generally  be  held  in  the  Fort  Detrick,  Maryland,  area,  but  may  occur  elsewhere  in  the  U.S. 
Participation  in  the  PLR  will  be  in  lieu  of  submitting  next  scheduled  Quarterly  report  required  under  the 
award. 


QUARTERLY  REPORTS 


a.  Quarterly  reports  are  the  most  immediate  and  direct  contact  between  the  Principal  Investigator 
(PI)  and  the  Grants  Officer's  Representative  (GOR).  The  reports  provide  the  means  for  keeping  this 
Command  advised  of  developments  and  problems  as  the  research  effort  proceeds.  The  quarterly 
reports  also  provide  a  measure  against  which  decisions  on  release  of  funding  and  on  requests  for 
supplements  are  made. 

b.  In  accordance  with  Section  C.,  a  Quarterly  Report  shall  be  submitted  for  each  three-month  period 
beginning  with  the  effective  date  of  the  assistance  agreement.  This  requirement  includes  all  three- 
month  periods  of  the  assistance  agreement. 

c.  Copies  of  each  report  shall  be  submitted  in  the  quantities  indicated  to  the  addresses  shown 
below  within  fifteen  (15)  days  after  the  end  of  each  quarter.  Internal  Government  distribution  will  be 
made  by  those  offices  (electronic  submission  preferred). 


(1)  Qne  (I)  copy  of  the  report  to: 

Grants  Qfficer's  Representative 

Military  Qperational  Medicine  Research  Program  (MQMRP) 

ATTN:  MAJ  Pedro  Bonilla-Vazquez 

MCMR-RTQ 

504  Scott  Street 

Building  722,  Room  32 

Ft.  Detrick,  MD  21702-5012 
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(2)  One  (1)  copy  of  the  report  to: 

Director 

U.S.  Army  Medical  Research  Acquisition  Activity 
ATTN:  MCMR-AAA-W  (W81XWH-09-2-0172) 

820  Chandler  Street 

Fort  Detrick,  MD  21702-5014 

d.  The  Quarterly  Report  sample  (See  following  Quarterly  Report  Format)  shall  serve  as  the  format. 
Each  item  of  the  report  format  shall  be  completed. 


QUARTERLY  REPORT  FORMAT 


1.  Award  No. _ 

2.  Report  Date _ 

3.  Reporting  period  from _ to 

4.  PI _ 

5.  Telephone  No. _ 

6.  Institution _ 

7.  Project  Title _ 


8.  Current  staff,  with  percent  effort  of  each  on  project. 
_ % _ 

_ % _ 


% 

% 


9.  Award  expenditures  to  date  (as  applicable): 

This  Qtr/Cumulative 

Personnel  _ / _ 

Fringe  Benefits  _ / _ 

Supplies  _ / _ 


This  Qtr/Cumulative 

Travel _ / _ 

Equipment _ / 

Other _ / _ 


This  Qtr/Cumulative 

Subtotal _ / _ 

Indirect  Costs _ / _ 
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Fee _ / 

Total _ / _ 


10.  Comments  on  administrative  and  logistical  matters. 


11.  Use  additional  page(s),  as  necessary,  to  describe  scientific  progress  for  the  quarter  in  terms  of  the 
tasks  or  objectives  listed  in  the  statement  of  work  for  this  assistance  agreement. 

12.  Use  additional  page(s)  to  present  a  brief  statement  of  plans  or  milestones  for  the  next  quarter. 

The  following  have  been  deleted: 

SPECIAL  CONTRACT  REQUIREMENTS 


Appendix  B:  Data  Linking  Procedures  and  Timeline 
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Army  Intervention  Data  Sources 

This  document  outlines  the  datasets  created  around  the  Army  Intervention. 

SM  SURVEY/PROVIDER  SURVEYS/PDHRA/PDHA/HCE  MERGE 

This  dataset  contains  data  for  every  Service  member  for  whom  any  of  the  following  is  true: 

•  Completed  a  Service  Member  Survey 

•  A  Provider  Post-PDHRA  3  Question  Survey  was  completed  by  their  provider 

•  They  were  in  the  linking  file  and  had  a  PDHRA  match  (this  happened  14  times,  likely 
because  occasionally  a  Service  member  would  complete  a  blue  card,  but  not  a  survey, 
and  the  provider  also  did  not  complete  a  3  Question  Survey  for  the  Service  member). 

The  following  subsections  outline  the  data  sources  that  comprise  the  larger  dataset. 

Service  Member  Survey 

Table  1.  Records  removed  and  total  records  for  Service  Member  Survey 

Total  Removed 

Raw  SM  Survey  records  n/a 

Records  deleted  if  Survey  does  not  represent  a  SM,  if  the  92 
survey  was  entered  by  QTSG,  or  if  the  survey  was 
duplicated 

Provider  Post-PDHRA  3  Question  Survey 

Table  2.  Records  removed  and  total  records  for  Provider  3  Question  Survey 

Total  Removed 

Raw  Provider  3  Question  Survey  records  n/a 

Records  deleted  if  survey  number  was  duplicated  5 

First  Provider  Survey 

Table  3.  Total  Records  for  First  Provider  Survey 
Number  of  First  Provider  Surveys 

Last  Provider  Survey 

Table  4.  Total  Records  for  First  Provider  Survey 
Number  of  Last  Provider  Surveys 

Workshop  Evaluation 

Table  5.  Total  Records  for  Workshop  Evaluation 

Total  Removed  Total  Records 

Number  of  Workshop  Evaluations  n/a  23 
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Total  Removed  Total  Records 

n/a  21 


Total  Removed  Total  Records 

n/a  38 


Total  Records 

3102 

3097 


Total  Records 
1608 
1516 
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Linking  File 

Table  5.  Total  Records  for  Linking  File 

Total  Removed  Total  Records 

Number  of  Linking  File  Records  n/a  1852 

PDHRA  (clean) 

This  is  a  tile  file,  and  has  multiple  records  per  Service  member.  It  includes  Army  Active  and 
Reserves.  Date  of  completion  ranges  from  10  Feb  2008  through  20  March  2012. 

Table  6.  Records  removed  and  total  records  for  PDHRA 


Total  Removed 

Total  Records 

Raw  PDHRA  Records 

n/a 

227,217 

Key  variable  missing  (date  departed  theater,  date  of  form 

227,215 

completion,) 

2 

Service  Branch  was  anything  besides  Army. 

310 

226,905 

Final  Number  of  PDHRA  Records 

226,905 

PDHA  (raw) 

Only  combat  exposure  variables  have  been  pulled  into  the  larger  data  set. 

Table  7.  Total  Records  for  PDHA 

Total  Removed 

Total  Records 

Number  of  PDHA  Records 

n/a 

373,431 

HCE  (raw) 

Table  8.  Total  Records  for  HCE 

Total  Removed 

Total  Records 

Number  of  HCE  Records 

n/a 

1,760,575 

ClinMerge 

This  is  the  compilation  of  the  above  datasets.  Table  9  shows  the  total  records  after  each  merge 

Table  9.  Total  records  at  each  step  of  the  merge 

Total  Removed 

Total  Records 

Merging  SM  Survey  x  3QX  merge 

n/a 

3212 

Merging  the  above  with  Other  Provider  Surveys 

n/a 

3212 

Merging  the  above  AFHSC  Linking  File 

n/a 

3244 

Merging  the  above  with  the  PDHRA 

n/a 

3273 

Records  if  there  were  multiple  PDHRA  matches  and  one 

44 

3229 

was  outside  the  intervention  dates. 

ALL  PDHA  &  PDHRAS  FOR  SMS  IN  THE  STUDY 

This  is  a  tall  file  that  includes  all  PDHRA  and  PDHA  (only  combat  variables  were  pulled  for  the 
latter)  records  for  Service  members  who  were  part  of  the  intervention.  The  criteria  for  being  "a 
part  of  the  intervention"  are  the  same  as  above: 
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•  Completed  a  Service  Member  Survey 

•  A  Provider  Post-PDHRA  3  Question  Survey  was  completed  by  their  provider 

•  They  were  in  the  linking  file  and  had  a  PDHRA  match  (this  happened  14  times,  likely 
because  occasionally  a  Service  member  would  complete  a  blue  card,  but  not  a  survey, 
and  the  provider  also  did  not  complete  a  3  Question  Survey  for  the  Service  member). 

There  are  two  variables  that  indicate  if  the  record  was  collected  in  relation  to  the  workshop: 
INTERVENTIQNPDHRA  and  INTERVENTIQNPDHA  (the  ladder  indicated  the  PDHA  that  is  matched 
with  the  PDHRA  related  to  the  workshop). 


Table  10.  Total  PDHA  and  PDHRA  records  for  all  Service  members  in  the  study. 

Total  Removed  Total  Records 

PDHA  Records  n/a  3036 

PDHRA  Records  n/a  2465 

Total  n/a  5501 


ALL  PDHRAS  COMPLETED  BY  THE  51  INTERVENTION  PROVIDERS 

This  is  a  tall  file  that  contains  all  the  PDHRAs  completed  the  51  providers  that  were  part  of  the 
intervention.  There  is  a  variable  called  INTERVENTIQNPDHRA  that  tells  you  which  PDHRAs  were 
completed  in  relation  to  the  workshop.  The  N  is  around  1700  and  indicates  a  linked  PDHRA 
that  was  completed  during  one  of  our  site  visits. 

Location  from  the  Provider/Location  Linking  file  has  been  imported  (variable  is  called  CITY),  but 
53.25  %  is  missing.  The  variable  LQCATIQN  is  from  the  site  visits,  and  is  complete. 

The  variable  WQRKSHQP  indicates  whether  the  provider  participated  in  the  workshop  during  a 
site  visit. 

The  variable  PREPQST  indicates  whether  a  PDHRA  was  completed  before  or  after  the  workshop. 
This  should  be  combined  with  WQRSHQP  to  determine  whether  the  provider  actually 
participated  in  the  intervention. 

Table  11.  Total  PDHRAs  for  intervention  providers. 


Total  Removed 

Total  Records 

Number  of  potential  PDHRAs  matched  to  Providers 

n/a 

26272 

Records  removed  if  event  date  in  the  linking  file  and  in 
the  Clean  PDHRA  file  were  not  within  7  days  of  each 
other. 

267 

26005 

Total 

n/a 

26005 
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Timeline  of  Data  Requests,  Data  Received,  and  Site  Visits 


PDHRA  Data  Requested  (all  PDHRAs  for  SMs  in  intervention  time  period  -  June  08,  2012) 


PDHRA  Data  Received  (Feb  2008  -  June  08,  2012) 


2008 


2009 


2010 


Oescriptives  for  dataset:  ALL 
PDHRAS  COMPLETED  BY  THE 
51  INTERVENTION 
PROVIDERS 

R:\Projects\Army\Data\DataS 

AS\NewData\AFHSC_Data\Cl 

eanData\DD2900_200801\Se 

rvice5_Data\ 

dd2900_withid_intervention. 

sas7bdat 


Ft  Campbell  (Oct.  19-31) 

N  PDHRAs:  8372 

N  PDHRAs  PRE  Intervention:  4758 
N  PDHRAs  POST  Intervention:  3614 
N  PDHAs:  8083 
N  HCE  6  weeks  prior:  16,718 
N  HCE  6  weeks  post:  21,909 

HCE  6  weeks  prior  per  SM:  2.0 
HCE  6  weeks  post  per  SM:  2.6 


Ft  Stewart  (Nov  1-8) 

Ft  Carson  (Nov  7-10) 

Ft  Bliss  (Mar  5  - 15) 

N  PDHRAs:  3909 

N  PDHRAs;  8721 

N  PDHRAs;  5003 

N  PDHRAs  PRE  Intervention:  2969 

N  PDHRAs  PRE  Intervention;  6060 

N  PDHRAs  PRE  Intervention;  3284 

N  PDHRAs  POST  Intervention:  940 

N  PDHRAs  POST  Intervention:  2661 

N  PDHRAs  POST  Intervention:  1719 

N  PDHAs;  3793 

N  PDHAs:  8442 

N  PDHAs:  4882 

N  HCE  6  weeks  prior;  6229 

N  HCE  6  weeks  prior;  19,281 

N  HCE  6  weeks  prior;  8605 

N  HCE  6  weeks  post:  10,450 

N  HCE  6  weeks  post;  35,732 

N  HCE  6  weeks  post;  13,476 

HCE  6  weeks  prior  per  SM;  1.6 

HCE  6  weeks  prior  per  SM:  2.2 

HCE  6  weeks  prior  per  SM:  1.7 

HCE  6  weeks  post  per  SM;  2.7 

HCE  6  weeks  post  per  SM:  4.1 

HCE  6  weeks  post  per  SM:  2.7 
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Appendix  C:  Presentation  Given  at  the  2013  American  Psychological  Association 

Annual  Meeting 
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Communication  Training  for 
Health  Care  Providers  to  Improve 
Military  Mental  Health  Screening 


SUSAN  DOUGLAS,  ANA  REGINA  VIDES  DE 
ANDRADE,  STEPHANIE  BOYD,  NICOLE 
FRAZER,  MELISSAFRAINE,  LYNN  WEBB, 
AND  LEONARD  BICKMAN 


APA  July  2  01' 


Q 

This  research  was  carried  out  in  cooperation  with  Force  Health 
Protection  and  Readiness  (FHP&R)  and  was  supported  by  the 
Department  ofDefense  (DoD  W81XWH-07-P-1026)  and  by  U.S. 
Army  Medical  Research  Acquisition  Activity  (USAMRAA 
W  8  iX  WH-  o  9-  2-  o  1 72) 


Man]j  thanks  to  the  countless  individuals  who  contributed  to  this 
pro Ject,  w  ith  sp  ecia  I  app  reciationfo  r  our  Serv  ice  Members  and 
th ose  who  pro  vide  careforthem. 
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Overview 


•  Rationale 

o  Health  screening  is  a  DoD  priority 
o  Findings  from  2009  evaluation 


•  Workshop  development 

o  Focus  on  patient-centered  communication 
o  Best  practice  approaches  to  training  included  emphasis  on 
experiential  learning 

•  Pilot  study 

o  Aims  and  methods 
o  Participants 
o  Results 

•  Conclusions  and  next  steps 


Rationale 
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Military  Periodic  Health  Screening 


Occurs  Throughout 
Deployment  Cycle 


Our  focus  on  PDHRA 


Post  Deployment  Health 
Re-Assessment 

Facilitates  access  to  care 
during  re -adjustment  3 
to  6  months  after  return 
from  deployment 

Evaluates  physical  and 
mental  health  symptoms, 
psychosocial  stressors, 
and  exposure  concerns 


I 
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Select  Evaluation  Findings  (2009) 


•  Overall  PDHRA  increases  access  to  care 

•  SM  self-report  responses  highly  consistent,  while 
provider  documentation  of  concerns  and  referrals 
was  quite  variable 

•  Substantial  minority  of  SMs  anonymously  admitted 
to  under-reporting  mental  health  problems  and  were 
not  identified  as  in  need  of  care 

•  Provider  -  SM  communication  patterns  indicate  a 
physician-centered  approach 

o  High  prevalence  of  closed -ended  questions,  re -statements 
o  Low  incidence  of  relationship -building  statements 


Workshop  Development 
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Patient-Centered  Communication 


•  Evidence-based  benefits 

o  Increase  disclosure  of  psychosocial  concerns 
o  Greater  compliance  with  treatment  recommendations 
o  Enhance  provider  satisfaction 
o  Effective  even  during  brief  encounters 

o  Communication  training  improves  skills  for  new  and 
experienced  medical  professionals 

•  Evidence-based  practice  improvement  strategies 

o  Experiential  learning 
o  Coaching  and  feedback 
o  Peer  support  and  discussion 


Half  Day  Workshop  for 

Experienced  PDHRA  Providers 

. - - 

Patient-Centered  PDHRA 
Model* 

z/ 

Specific  Strategies  Covered 

On«nt4iV« 

toInhjnnClHctmn 

M  Vur  Swmary  [  1 

\  Hew  J  % 

•  Building  therapeutic  alliance 
o  Active  listening 

o  Empathizing 

•  Active  partnering 

o  Summarize  using  SMs  language 
o  Ask  for  preferences /goals 
o  Check  for  understanding 

•  Psychoeducation 

o  Normalizing  concerns 
o  Legitimizing  treatment  options 
o  Offering  resources 

•  Tailored  existing  model  to 
PDHRA 

o  Context  free  behaviors  fe  5., 
asking  open-ended  questions) 
o  Context  specific  behaviors  (e.g., 
brief  counseling  on  comnaonre- 
integration  concerns) 

*  aasnf  CPI  th€  Macp:  -.//T^rHoc^Tnedn^  £i^/ai7Ticiiiu7n/n\o(Sci/7Haao>ifm>  || 
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Audio  case 
examples 


Key  Words  at  Key  Times 


Why  U  it  impo«t4nl  to  r«p4n«J  on  the  Soldier's  DD  Form  iW)  rrspome*? 
lncfM«r  Ihr  opponuraff>'  torioMwr  Jn<k>*wrr^^  Kc<cic<||pMK<n  oT  IV(44em^^  Krfm^d 


Based  on  recording  of 
actual  PDHRA 
enoountETS 

Re-enacted  by  actors  to 
protect  patient 
confidentiality 

Selected  to  fedlitate  peer 
discussion  of  specific 
communicatian  skills 


How  do  you  expjnd  on  the  Soldier'*  DD  Form  2900  mtponse*? 


© 

Key  Word*  at  Key  Times: 


•  ’Tell  me  more  about  tKat...* 

•  “For  some  Soldiefx  a  re»pon>r  to _ may  indicate _ _  Uhar  led  )ou 

to  marl 

•  “Let  me  sumnun/e  what  I  heard.  WKal  elte^ 

•  *SomrSoldieni  find  H  dilfkull  to  r*pma  their  concerns  on  the  fcimn.  In 
gimcral  what  Kib  most  owKemed  you  since  your  return  hotneT* 

1_ A«*  llWTK  tunlunt  4w(  MfitW  hnKk  onKftnt  txj  twfifrt 


Interspersed  throu^ 
firstahoureof 
facilitated  discussion 


Emphasis  on  Experiential  Learning: 
Peer  discussion  of  audio  snippets 


Simulated 

Patients* 

4  cases  developed  using 
multiple  methods 
(reviewof  actual 
recorded  enoountere , 
case  study  presentatians 
by  military 
psycho  lo^sts ,  input 
from  expert  military 
panel) 

Delivered  ranotely 
through  video 
technology  during  last  2 
hours  ofvwrkshop 

Group  training  fbnnat 
fbr  peer  en^^ment  and 
immediate  feedback 


framing  and  manapeTHentjfrcvfdedi^  VUMCCrtferfor  Exiterienfraf  Caorrmp  /■Program  «  /^mcn 


Emphasis  on  Experiential  Learning: 
Interaction  with  Simulated 
Patients 
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Pilot  Study 


RESEARCH  AIMS 
METHODS 
PARTICIPANTS 
RESULTS 


Pilot  Study:  Aims  and  Methods 


•  ResearchAims 

1.  Explore  providers’ 
perceptions  ofworkshiop, 
acceptability  and  feasibility 

2.  Assess  impact  of  workshop 
on  soldiers’  ratings  of 
provider  communication 
skills 

3.  Determine  efficacy  of 
workshop  on  identification 
of  soldiers’  psychcoodal 
concerns 

•  Design 

o  Quasi-experimental 

o  Multi-daydata  collection  with 
workshop  at  midpoint 

o  Naturally  occurring 
comparison  group  of  non- 
workshop  providers 


o  Pre- and  pcst-workshop 
provider-report  measures  of 
communication  self-efficacy 
o  Surve5s  completed  after 
PDHRA  encounters  by  soldiers 
and  providers  about  the 
encounter  and  related 
attitudes 
•  Analyses 

o  Mixed  models, indudine 
poissonloglinear  modeT 
o  Clustered  by  provider  flCC 
595**)  not  bylocation  (ICC  x%) 


♦p  <  0.05;  **p  <  0.0 1;  <  0.00 1 
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Participants 


•  Four  installations 

o  Site  visits  ranged  from  4  to  9 
days 

o  9  to  i6  providers  per  site 
participated  in  data  oollection 
Cn=5iJ 

o  About  halfthe  providers  at  each 
site  participate  in  workshop 
Cn=233 

o  3  750  total  PDHRA  encounters 

o  i366cases  with  all  data 


encounters 

Provider  Survc>5 

Surveys 

yOi 

:c4i 

ZOi 

UZS 

9lS 

4l5i 

ftrant  Rarticiptir^  Proukfers 

tVortshop 

non- 

lAlorlstop 

ri^fc  genler 

60h 

Age  3049  Yean 

PiTfSiebnASS  BiBITt 

Current  or  Ibrncr  military 

70»i 

G  E  3  yea  n  PDHPA  e<.|K  rb  ns 

4:h 

i3h 

GE  11  yean  HCPenferbns 

Th 

40M 

Providers  were  typically  PAs, 
male,  30  -  50  years  old 
Workshop  providers  may  have 
been  more  experienced 


Results 

Aim  1:  Explore 
providers’  perceptions 
of  workshop 
acoeptabilitvand 
feasibilitv 

Measure:  Post¬ 
workshop  evaluation 
survey(devebped  for 
this  stufy) 


Providers  found  the  workshop  useful 
and  relevant  to  PDHRA 


100%  agreed  or  strongly  agreed 

o  Techniques/materials  usefuland  trainers 
were  knowledgeable/effective 
o  Content  wasrelevant  to  their  work,  would 
help, them  be  more  effective  in  eliciting 
soldiers’  mental  health  concerns,  intended  to 
try  using  the  communication  skills  in  practice 

Barriers  to  feasibility 
o  5  2  96  concerned  that  communication  skills 
would  increase  PDHRA  encounter  length 
(note:  when  asked  at  end  of  site  visit,  j6%  of 
providers  reported  no  or  minor  acceptable 
increase  in  time) 


116 


Annual  Report:  Contract  #W81XWH-09-2-0172 


Results 


Aim  3:  Assess  impectof 
workshop  onsoldieis' 
ratings  oTpoovider 
commtmicadon  skills 

Measure:  Communication 
Assessment  Tool  (CAT; 
Makoul  etal,2007) 

Modified  from  is  to  14 
items  (a  =  0.9® 

Sample  items 

This  provider... 

Treated  me  withrespect 

Let  me  talk  without 
intem^ttions 

Involved  me  in 
decisions  as  much  ^  I 
wanted 


<  0.05;  **p  <  0.0 1;  <  0.00 1 


According  to  soldiers,  the  WO rksh op 
increased  provid  ers’p  o  f  i  en  f-  centered 
communicationskiUs 

Soldiers*  RfltiD^s  of  Provider  CoimruDicatiou 
Skills;  Prc'OTjdPost'Worksbc^ 


‘tVerUep- 


Non-workshop  providers  were  rated  similarly  on 
patient-centered  communication  before  and  after 
Workshop  providers  were  rated  significantly 
higher  after  the  workshop  compared  to  before  (t  = 
2.7**) 


There  was  no  main  effect  for  workshop  on 
provider  identification  of  soldiers' MH  conceiTis 


Results 

Aim  3:  Determine  efScacy 
of  workshop  on  providers' 
identi£cationof  soldiers' 
psychosocial  ooncems 

Measure:  VU-developed3 
item  survey  completed 
immediatdy  after  the 
PDHRA  encounter 


The  majorityof  PDHRA  encounters  were  rated  as 
havingno  identification  ofMH  concerns 

o  Mean=o.6i(ran^ot0  3) 
o  64%  had  a  score  of  zero 

Know  from  past  research  the  maj  orityof  soldiers  do 
not  have  anyproblems  to  identify;  need  a  covariate 
Soldiers  reported  anonymously  whether  they  or  others 
(e  .g . ,  friends)  thought  they  had  a  MH  concern 


Seale  as sumofitemsCa  = 
0.74) 


Auotjj'iiious  soldier  report  of  se  If-  or  otber- 
identified  MH  coiicerDS 


Any  disciisionof  MH 
concetrs  not  alreai^ 
reported? 

Think  the  soldier 
disclosed  all  MH 
symptoms? 

Beliete  the  soldier  could 
benefit  fiom  further 
e  valuationfor  MH 
symptoms 
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Results 

Aim  3  Detennine 
efficacy  of  workshop  on 
providers’  identification 
of  soldiers’  psvchosocial 
concerns 


Among  soldiers  with  anonymously  self-reportee 
MH  concerns,  the  workshop  increasedprouider 
identification  of  soldiers’ MH  concerns 


Reiulti  oT  Pol  lion  Loglln«3r  R»gr»iilon 

EitmaUng  Rxivld^r  ld«n1l1lcaton  ofMHOoncemi 

psramster 

Eitmat^ 

Std.  BTor 

Z 

hcld^nc^ 
Rat»  Rato 

-1.1 

□2 

-62 

□-34 

Atoite  kcp 

□.1 

D2 

rin«  (pre -post 

0^ 

D2 

i.i 

\^i 

SoMIe  r-Me  iWled  M  H  ccic«  rts 

12 

□2 

319 

/UoiKstop  'Tln« 

-o;3 

□2 

-l.i 

□.72 

AKXKSlOp  'COIC«(li 

QJQ 

□0 

-0.1 

Dy 

rin«  '  Coiotrii 

-G.i 

□2 

-2j6^ 

□57 

/Uoilstop  'Tln«  'Coic«ris 

0j6 

Q3 

2^ 

153 

o  strong  positive  association  between  soldiei^identified  MH 
concerns  and  provider  identification  of  HH  concerns 
o  Bigger  drop  in  provider  identification  of  soldier  MH  concerns 
pre-  to  post-workshop  for  soldiers  who  self-identified  with 
MH  concerns  compared  tosoldierswithnoMH  concerns 

increase 
Qy  where 

5  anonymously  reported  MH  conoems 

o  1.83  times  greater  than  similar  soldiers  who  reported  MH 
concerns  but  dieirnroviders  did  not  receive  tKe  vorksKon 


♦p  <  O.OJi  **P  <  0.0 1;  <  0.00 1 


Conclusions  and  Next  Steps 


•  Half-day  workshop  developed  that  provided 


o  Patient-centEredc(mrimuriication  skills  tailored  to  military  health  screening 
process 

o  Heavyemphasis  on  experiential  leamingtechniques 
o  Appropriate  level  of  trainingfbr  experienced  professionals 

•  Workshop 

o  Found  to  be  acceptable  and  feasible  by  participating  providers 
o  Increased  providers’  patient-centered  communicatiDn  skills  as  rated  by  soldiers 
o  Increased  providers’  identification  of  MH  conoems  through  discussion  and 
evaluation  -  only  inoontextofPDHRA  encounters  where  soldiers  anonymously 
reported  MH  conoems 

•  Limitations 

o  Quasi-experimentalpilotstudywithno  randomization 
o  Small  sample  size  (siproviders) 

•  Next  steps 

o  Analvsis  of  time-series  data  from  electronic  healthreoords 
o  FinalreportduetoUSAMRAA  this  fall 
o  Disseminate  findings  within  DoD  communityof  interest 
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Appendix  D:  Presentation  Given  at  the  Force  Health  Protection  and  Readiness 

Psychological  Health  Forum 
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Communication  Training  for 
Health  Care  Providers  Improves 
Military  Mental  Health  Screening 


SUSAN  DOUGLAS,  ANA  REGINA  VIDES  DE 
ANDRADE,  STEPHANIE  BOYD,  NICOLE 
FRAZER,  MELISSAFRAINE,  LYNN  WEBB, 
AND  LEONARD  BICKMAN 


Psych  Health  Forum  Sept  2013 


0 

This  research  was  carried  out  in  cooperation  with  Force  Health 
Protection  and  Readiness  (FHP&R)  and  was  supported  by  the 
Department  ofDefense  (DoD  W81XWH-07-P-1026)  and  by  U.S. 
Army  Medical  Research  Acquisition  Activity  (USAMRAA 
W  8  iX  WH-  o  9-  2-  o  1 72) 


Many  thanks  to  the  countless  individuals  who  contributed  to  this 
pro ject,  w  ith  sp  ecia  I  app  reciationfo  r  our  Serv  ice  Members  and 
those  who  pro  vide  careforthem. 
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Overview 


•  Rationale 

o  Health  screening  is  a  DoD  priority 
o  Findings  from  2009  evaluation 


•  Workshop  development 

o  Focus  on  patient-centered  communication 
o  Best  practice  approaches  to  training  included  emphasis  on 
experiential  learning 

•  Pilot  study 

o  Aims  and  methods 
o  Participants 
o  Results 

•  Conclusions  and  next  steps 


Rationale 


121 


Annual  Report:  Contract  #W81XWH-09-2-0172 


Military  Periodic  Health  Screening 


Occurs  Throughout 
Deployment  Cycle 


Our  focus  on  PDHRA 


Post  Deployment  Health 
Re-Assessment 

Facilitates  access  to  care 
during  re -adjustment  3 
to  6  months  after  return 
from  deployment 

Evaluates  physical  and 
mental  health  symptoms, 
psychosocial  stressors, 
and  exposure  concerns 


PDHRA*:  Two  components 


Self-Report  Form 


Brief,  typically 
completed  online 
Specific  questions  ask 
about  4  areas  of 
psychosocial  concern 
o  Relationship  conflicts 
o  PTSD  symptoms 
o  Alcohol  use 
o  Depression 


Health  Care  Provider 
Interview 


Brief  (average  10  min.) 
Provider  interviews  goals 
o  Review  self-r^rt  responses 
and  provide  a  brief  health 
assessment 
o  Provide  education 
o  Make  referrals  for  further 
evaluation  as  warranted 

Atypical  medical  encounter 

o  No  diagnosis, prognosis, or 
treatment  maybe  given 
o  No  physical  exam 
o  SM  required  to  participate 


*4z  conducted  at 


tmcfude  adt^hanaf  ngntaf  hgafthasez^entz. 
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15  Would  yov  to  »cMdal»  »  with  »  proind«f  to  you*  O  Yo«  O  No 

cofWonMo)^ 

15.  An  you  curroftdy  wilorootod  In  roooivinj  Mtofmatlofi  or  ooorotonoo  for  o  otrooo.  oomMIoaoI  or  q  ^ 

oleohol  ooneom'? 


Select  Evaluation  Findings  (2009) 


•  Overall  PDHRA  increases  access  to  care 

•  SM  self-report  responses  highly  consistent,  while 
provider  documentation  of  concerns  and  referrals 
was  quite  variable 

•  Substantial  minority  of  SMs  anonymously  admitted 
to  under-reporting  mental  health  problems  and  were 
not  identified  as  in  need  of  care 

•  Provider  -  SM  communication  patterns  indicate  a 
physician-centered  approach 

o  High  prevalence  of  closed -ended  questions,  re -statements 
o  Lovf  incidence  of  relationship -building  statements 


http:, 'handle  jj  tic  jruI/iQ  0:2/  filjA^.‘2Qo6?, 
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Workshop  Development 


Patient-Centered  Communication 


•  Evidence-based  benefits 

o  Increase  disclosure  of  psychosocial  concerns 
o  Greater  compliance  with  treatment  recommendations 
o  Enhance  provider  satisfaction 
o  Effective  even  during  brief  encounters 
o  Communication  training  improves  skills  for  new  and 
experienced  medical  professionals 

•  Evidence-based  practice  improvement  strategies 

o  Experiential  learning 
o  Coaching  and  feedback 
o  Peer  support  and  discussion 
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Half  Day  Workshop  for 
Experienced  PDHRA  Providers 


Tailored  existing  patient -centered  model  to  PDHRA* 


Workshop  Structure  and  Content 


Structure  Sample  Content 


•  Tailored  existing  model  to 
PDHRA 

o  Context  free  behaviors  (e  5., 
asking  open-ended  questions) 
o  Context  specific  behaviors  (e.g., 
brief  counseling  on  common  re¬ 
integration  concerns) 

•  Emphasis  on  experiential 
learning 

o  Didactic  presentation 
o  Audio  case  examples  for 
facilitated  discussion 
o  Simulated  patient  interviews 


•  Building  therapeutic  alliance 
o  Active  listening 

o  Empathizing 

•  Active  partnering 

o  Summarize  using  SMs  language 
o  Ask  for  preferences /goals 
o  Check  for  understanding 

•  Psychoeducation 

o  Normalizing  concerns 
o  Legitimizing  treatment  options 
o  Offering  resources 
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Simulated 

Patients* 


4  cases  developed  using 
multiple  methods  (review 
of  acti«l  recorded 
encounters,  case  stiify 
presentations  by  military 
psychologists ,  input  from 
expert  rriktaty  panel) 

Delivered  ranotelyusing 
video  technolQgyduring 
last  2houi5  of  workshop 

Group  training  fbrmatfbr 
peer  en^^ment  and 


Emphasis  on  Experiential  Learning: 
Interaction  with  Simulated 


It 


immediate  feedback 


Patients 


SP  framm^  and  mana^^eTHgntjfratjrdcdb^  VUMCCenterfor  ^x]^erkmfiaf  /’Program  «  ^mcn 
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Simulated  patier 


•  Private  First  class  Adam 
Davidson 

o  Age:  20 
o  RankiEs 
o  Mar ital  Status  :Singb 
o  Deployments:  1  to  Afehanistan 
o  SP#i  “Barriers  to  care” -young 
enlisted  soldier’s  conoems  about 
stigma  and  confidentiality 

•  Sergeant  Robert  Woodson 
o  Age:  27 

o  Rank:Es 

o  Mar  ital  Status:  Married 
o  Deployments:  2  to  Aighanistan 
o  SP#2  “Avoidance  and  denial” - 
soldier  to  recognize  his  mental 
health  symptoms 


Cover  sheet  and  completed  DD  Form  2900  pro' 


Simulated  patients 


•  Colonel  Diana  Perez 
o  Age:  40 

o  Rank:  0  5 

o  Marital  Status:  Single 
o  Deployments:  2  to  Iraq 

o  SP  #  3  “Career  concerns”  -  goal  to  elicit  disclosure  from  high-ranking 
soldier  who  is  reluctant  to  talk  about  her  health  concerns  due  to 
conoems  about  impact  on  career 

•  Sergeant  David  Thompson 
o  Age:  40 

o  Rank:  E  7 

o  Marital  Status:  Separated 
o  Deplo5mients:  3  total,  1  to  Iraq,  2  to  Afghanistan 
o  SP  #  4  “T  ime  management”  -  soldier  presents  as  tangential  and  difficult 
to  manage  as  he  talks  about  personal  problems  jprovider  must  maintain 
control  or  the  interview  to  complete  in  a  timely  fashion 


Cover  sheet  and  completed  DD  Form  2900  provided  in  hinder 
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Pilot  Study 


RESEARCH  AIMS 
METHODS 
PARTICIPANTS 
RESULTS 


•  ResearchAims 

1.  Explore  providers' 


2.  Assess  impact  of  workshop 
on  soldiers’  ratings  of 
provider  communication 
skills 

3.  Determine  efficacyof 
workshop  on  identification 
of  soldiers’  psychcoodal 
conoems 

Design 

o  Quasi-experimental 

o  Multi-daydata  collection  with 
workshop  at  midpoint 

o  Naturally  occurring 
comparison  group  of  non- 
workshop  providers 


♦  p  <  0.05;  **p  <  0.0 1;  *^H(p  <  0.00 1 


o  Pre- and  pcst-workshop 
provider-report  measures  of 
communication  self-efficacy 
o  Surveys  completed  after 
PDHRA  encounters  by  soldiers 
and  providers  about  the 
encounter  and  related 
attitudes 

o  Electronic  health  records 
including  PDHRAs 

Analyses 

o  Mixed mod^, including 
poissonloglinear  modeT 
o  Time  series  (proc  panel) 
o  Clustered  by  provider  CICC_5% 
to  io3S  **)  not  by  location  CttC 
1%) 
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Participants 

Installation  and  Provider  Characteristics 


•  Four  installations 

o  Site  visits  ranged  from  4  to  9 
days 

o  9  to  i6  providers  per  site 
participated  in  data  oollection 
Cn=5iJ 

o  About  halfthe  providers  at  each 
site  participate  in  workshop 
(workshop  n= 23) 
o  3  750  total  PDHRA  encounters 
o  i366cases  with  all  data 


1  InstAlbtlun 

PDHRA 

Bnc«untcrs 

Prcvl^lcr 

Survc>4 

2«ldl«r 

1 

2 

710 

:ch 

39H 

1 

A 

giS 

4lS 

“L _ _ _ _ _ _ _ J 


ftrant  Rarticiptir^  Pmkfers 

iVortshop 

non- 

lAlorlstop 

rvtafc  genler 

60h 

Age  3049  YCBR 

P^TfsicenAss  stBirt 

7*h 

G7h 

Current  or  Ibrncr  military 

70»i 

G  E  3  yea  n  PDHPA  e«.|K  re  ns 

4:h 

i3h 

GE  11  yean  HCPenfercns 

Th 

40M 

Providers  were  typically  PAs, 
male,  30  -  50  years  old 
Workshop  providers  may  have 
been  more  experienced 

Selection  non -random 
(convenience  sample) 


Participants 

Soldier  Characteristics 


o  41%  18-24  years 
o  29%  25-29  years 
o  2496  30-39  years 
o  6%  40  years  and  over 

•  9496  male 

•  All  active  duty 

„  ,  •  No  differences  across  groups 

•  Rank 

o  58%  El  -  E4 
o  28%  E5  -  E6 

O  596E7-E9  : 

o  9%0i-09,  W01-W05 
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Aim  1 


Results 

Aim  i:  Explore 
providers’  peraeptiotrs 
of  workshop 
acceptability  and 
feasibility 

Measure:  Post¬ 
workshop  evaluation 
suivey(devebped  for 
this  stufy) 


Providers  found  the  workshop 
useful  and  relevant  to  PDHRA 

•  100%  agreed  or  strongly  agreed 

o  Techniques/ materials  useful  and 
trainers  were  knowledgeable/effective 

o  Content  was  relevant  to  their  work 

o  Would  help  them  be  more  effective  in 
eliciting  soldiers’  mental  health 
concerns 

o  Intended  to  try  using  the 
communication  skills  in  practice 
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Results 

AimiiExpbre 
providets’  perceptions 
of  workshop 
acceptability  and 
feasibility 


Even  experienced  providers  felt 
that  the  workshop  fulfilled  often 
unmet  provider  needs. 

•  “This  was  very,  very  valuable.  I 
have  been  doing  PDHRAs  for  over 
5  years  and  would  have  benefited 
from  this  at  my  initial  training.  A 
yearly  refresher  for  all  providers 
would  also  be  valuable.  Hearing 
the  input  from  my  colleagues  was 
particularly  useful.” 


Results 

Aim  i:  Explore 
providers’  perceptions 
of  workshop 
acceptability  and 
feasibility 


Providers  liked  the  experiential 
learning  and  peer  interaction 

•  Providers  expressed : 

o  Frustration  at  the  “slide  deck”  mode  of 
training  typically  used  at  their  facilities. 

o  Strong  support  for  simulated  patients  as 
usefultraining  tools. 

o  Appreciation  for  the  opportunity  to  learn 
how  colleagues  dealt  with  difficult 
interview  situations. 

K  AtallsiteSjpr  orders  said  they  rarely  had 
such  opportunities. 

K  During  discussion,  providers  often  wrote 
down  /said  they  wanted  to  try  techniques 
that  their  colleagues  described  using. 

o  Several  providers  stated  they  wanted  more 
training  and  feedback  opportunities 
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Results 

AimiiExpbre 
providers’  perceptions 
of  workshop 
acceptability  and 
feasibility 


Simulated  patient  interactions 
and  group  discussion  were  the 
most  widely  valued  workshop 
components 


Most  Useful  Workshop  Component 


Simulated  patient  interactions  57 

Group  discussion  39 

Any  specific  communication  strategy  13 

General  discussion/model  of  communication  1 3 


«Total  is  over  100%  because  some  provideis  Isted  more  thanone 
"most  useful"  component 


1 - 

■  ■  1 

w 

Providers  mentioned  structural  barriers 
to  feasibility 

Results 

0  Concernthat  use  of  communication  skills 

Aim  1:  Explore 

would  increase  PDHRA  encounter  length 

providers’  perceptions 

X  5  2%  immediatelyafter  the  workshop  said  yes 

of  workshop 

K  yd^reporlednoorminoracoeptable  increase  in 

acceptability  and 

team  at  the  end  of  the  site  visit 

feasibility 

0  2  2  96  felt  they  would  not  be  supported  in  using 

Measures:  Post- 

the  skills  by  installation  leadership 

workshop  evaluation 

0  Concerns  about  structural  problems  with 

survey,  Post-workshop 

installation  capacity  to  handle  referrals: 

self-efficacy  survey 

X  Fear  of“opening  up  psychissues  and  motivating 
soldiers  to  seek  care  when  the  resources  are  not 
able  to  aanmmodate  themin  a  timely  basis 

X  Encounters  withsoldiers  who  had  alreadybeen 
frustrated  by  long  waits  for  short  appointments . 

■ 
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Aim  2 


© - 

ASSESS  IMPACT  OF  WORKSHOP  ON 
SOLDIERS’  RATINGS  OF  PROVIDER 
COMMUNICATION  SKILLS 


SM  Survey  Scales:  Provider  Ratings 


Comnuniatbn  ABSBnc(TtTo0l(CAT) 

Rem 

Mean 

Std.  Cev.| 

CombrlBble  greetiif 

4.1 

10 

Ticqlrd  with  rt{|rct 

43 

03 

Irrtertst  in  ideas 

4.1 

10 

Unlentocd  consrn 

4.1 

10 

nid  attention 

4.1 

1.1 

*8  Ik  wo  interruption 

42 

03 

Enoif  h  inlOrnstion 

4.1 

10 

Easy  to  urdentaiil 

42 

03 

Ctccicd  uiderstBiilirf 

4.1 

10 

Ecouraged  qicstiora 

33 

12 

Imrohied  indecaion 

40 

1.1 

Dbcies  icAtste|s 

40 

1.1 

ShoMd  oonarn 

40 

1.1 

SfsrTt  r^lrttinc 

40 

1.1 

Communication 

Assessment  Tool  (CAT) 

•  Construct  (context  free): 
provider’s  patient-oentered 
communication  skills 

•  14  items  retained  from  original 
15  (Makoul  et  al.,  2007) 

•  Cronbach’salpha  =  o.98 

•  Responses  range  from  1  (poor) 
to  5  (excellent)  with  higher 
scores  better 

•  Overall  scale  mean  =  4.1  (s.d.= 
0.9) 


Makoul,  Krupat  et  al,  2007 
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SM  Survey  Scales:  Provider  Ratings 


FtTfSican^  Hurninstic  BsraYbn  Q^stbnnine  IPHBQJ 


Misan 

Std.  Cev.| 

Ino  hurfy  irevenedl 

33 

07 

^pressed  conarnbr  Irelir^s 

33 

07 

Asked  hOM*  i'MBs  doirf 

3A 

07 

h>tode  umrirf  rernsrla  Ireurnedl 

37 

Oi 

Short1ern|Ered  oredrupt  Ircurrsedl 

37 

Oi 

KnoY^fedgeabfe  about  poctbeploynerTt 
conzrra 

33 

07 

Phusician^sHum  a  nistic 
Behaviors  Questionnaire 
(PHBQ) 

*  Construct  (context  free): 
providei^s  patient-centered 
communication  skills 


6  items  retained  from  original 
25  (Weaver  et  al.,  1993)  with 
modified  wording 
Cronbach’s  alpha  =  0.83 
Responses  range  from  1 
(Strongly  disagree)  to  ^ 
(Strongly  agree)  with  higher 
scores  Detter 


•  Overall  scale  mean  =  3.4  (s.d.= 

0.5) 


Weaver, Ow,etal,  1993 


SM  Survey  Scales:  Provider  Ratings 


PDHRA  intenrew  ftiBvbn  |PIS| 

frtoan 

Stil.  CSV.] 

Asked  about  pb^sial  icatth 

IjO 

03 

Asked  BboutemotbnBi  rcatth 

03 

03 

As  led  a  bout  a  loohol  ise 

07 

0.5 

kfelp  nc  reocfna  probfena 

07 

0.5 

IbU  rre  about elfecthie  treatnents 

07 

0.5 

Gaw  advice  on  bow  to  access  cete 

07 

04 

*8  Iked  about  common  leintegretbn  asiss 

0.5 

0.5 

Asked  triwas  almady  teatnent 

lOr  reported  oonceria 

03 

03 

Asked  nc  howsatalted  Iwaswitki 
treatnerTtlor  reported  concerre 

07 

0.5 

^PDHRA  Interview 
Behaviors  (PIB) 

•  Construct  (context  specific): 
expected  provider  behaviors 
during  PD HRA  interview 

•  9  items  developed  for  this 
study 

•  Cronbach’s  alpha  =  0.87 

•  Responses  of  yes  (1)  or  no 

(o) 

•  Overall  scale  calculated  as 
the  sum  of  items;  mean  = 
6.0  (s.d.  =  2.5) 


VU  derived  scale 
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According  to  soldiers,  the  wo rksh op 
increased  provid  ers’p  atient-  centered 
communicationskills 


Results 

Aim  2:  Assess  impact  of 
workshop  on  soldiers’ 
rating  of  provider 
communication  skills 


Reiulti  oT  Hl»nn:lilcal  Linear  Model  Regreiiloni 

Eitlmatng  Soidien*  RaUnai  of  PDHRA  ntervlev/ 

M»a  luie 

Eitmate 

Std.  Bnor 

■A-aid’iz 

Oolmn'i  C 

D? 

0.1 

2j6* 

□29 

PHBO 

D.1 

0,1 

- 

PIB 

□-If. 

0.3 

2V 

□23 

fnfgr^retQtiarx  Cohgn'^( {ioTQz) 


All  analytic  modd$ 
contrdled jbr 
cKistmngvXtHn 
providers  and  mdn 
effects  eftime  cs^d 
workshop. 


•  For  the  CAT  and  PIB 

o  Workshop  providers  were  rated  significantly  higher  after  the 
workshop  compared  to  before 

o  Non-workshop  providers  wereratedsimilarlybefore  andafter 

•  Variance  attributable  to  providers  as  measured 
with  intraclass  correlation  coefficients  (ICCs) 
indicated  medium  effects 


O  CAT  ICC  =12% 
O  PHBQICC=9% 
O  PIBICC=iO% 


Tnht} (iiaudff\iKi3h  ^  Cu  (saaa):  ^  {^aft),  rOfi  (ngdiun),  (kirgg) 


*p<  O.OJi  **P  <  0.0 1;  <  0.00 1 


Aims 
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Results 

Aim  3:  Determiie 
of  workshop  on  providers' 
identi£cationof  soldiers' 
psychosocial  ooncerrE 

Measure:  VU-developed3 
item  svirvey  completed 
immediatdy  after  the 
PDHKA  encounter 

Sc<r/e  as  stjmofitzTns(a  = 
0.74) 

AnydisctEsionofMH 
concern!  not  alreaify 
reported? 

Think  thE  soldier 
disclosed  all  MH 
symptoms? 

Beliete  the  soldier  could 
benefit  fiom  further 
e  valuationfor  MH 
symptoms 


There,  was  no  main  effect forworkshop  on 
provider  identification  of  soldiers' MH  concerns 

•  The  majorityof  PDHRA  encounters  were  rated  as 
havingno  identification  ofMH  concerns 

o  Mean=o.6i(ran^ot0  3) 
o  64%hadasco(reofretD 

•  Know  from  past  research  the  maj  orityof  soldiers  do 
not  have  anyproblems  to  identify;  need  a  covariate 

•  Soldiers  reported  anonymously  whether  they  or  others 
(e  .g . ,  friends)  thought  they  had  a  MH  concern 

soldier  report  of  sc  If-  or  otber- 
idciJtified  VTH  coTJCcros 


Among  soldiers  with  ononi/mousl^  self-reportec 
MH  concerns,  the  workshop  increasedprovider 
identification  of  so/diers'Mff  concerns 


R«iulti  ofPoliion  Loglln«sr  R^greiilon 

Eitimstlng  R^vld^r  u^ntfleaton  orMHConc^mi 

P3r?m»t»r 

Eitmate 

Std.  Bror 

■A'^WI  Z 

ncio»nc« 
Rat»  Rato 

lifercept 

-1.1 

□2 

-62 

□.3i 

Atoitekop 

□.1 

DjS 

rin«  (pre-posti 

D2 

1.1 

3oMle(-Meiinied  MHcotcens 

12 

□2 

AloiKskop  'Thie 

-o;3 

□2 

-l.i 

□  72 

AtoiKttop  'Coicerii 

Oil 

-0.1 

rin«  ‘  Coiotri! 

-€.i 

□2 

-2j6^ 

□5? 

AloilsIOD  *Tlne  'Coicerii 

n.6 

0.? 

2.7* 

o  strong  positive  association  between  soldier-identifiel  MH 
concerns  and  provider  identification  of  MH  concerns 
o  Bigger  drop  in  provider  identification  of  soldier  MH  concerns 
pre-  to  post-workshop  for  soldiers  whoselfidentified  with 
MH  concerns  compared  to  soldiers  with  no  MH  concerns 

__  increase 
ly  where 

anonymously  reported  MH  conoems 

o  1.83  times  greater  than  similar  soldiers  who  reported  MH 
concerns  but  their  pro  viders  did  not  receive  the  workshor 
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E 


Results 

Aim  3  Detennine 
efficacy  of  workshop  on 
providers’  identification 
of  soldiers’  psychosocial 
concerns 

Electronic  health 
records  -  all  26,005 
PDHRAsfbrthe^ 
providers  for  the  tirre 
period  of  2Qi±to 

December  2013. 


Panel  Analysis  :  Difference  in  Difference 

Difference  in  Difference  (DD)  compares  the  changes 
in  outcomes  overtime: 

DD=[Time  Chan^  Workshop  Group]  -  [Time 
Change  Comparison  Group] 


Integrates  2  elements 

o  Time  in  days:  befcre  and  after  the  worlshop  for  each 
provider 

o  Treatrr»l  group  (participation  in  vcrkshop) 

Every  provider  keeps  own  calendar:  keeping  all 
time  points  and  all  PDHRAs  reported  each  time 
Compares  the  average  change  by  treatment 
group  in  two  time  segments 
Strength:  it  accounts  for  providers’  differences 
between  groups  that  are  constant  over  time 


Workshop  Effect  =(A-B)-(C-D) 


Providers’  Outcomes  =FCTime,Workshop,SM’s  Characteristics, Time*Workshop) 
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There  were  2381  28 
providers  in  the 
workshop /oomparison 
group  respectively. 

Providers  participating 
in  the  workshop: 

1)  392PDHRAS  before  & 

1 96  PDHR  As  after  the 
workshop. 

2)  Data  on  202davs  before 
8c  isSda^  after. 


Sample  Characteristics 


Providers'  Averages 

Group 

1  Tune  1 

Before  After] 

SMs  X  Provider 

Workshop 

392  196 

No  Workshop  288 


AvgTime  in  Daj®  x  Providerworkshop 


No  Workshop  206 


158 

158 

119 


Providers  without  participating  in  the  workshop: 

1) 288  PDHRAs  before  &  1 58  PDHRAs  after  the 
workshop. 

2)  Data  on  2o6daj5  before  &  ii9days  after. 


The  DD  estimated 
workshop  effect  on  3 
providers’  outcomes 
are  reported  here. 

For  each  outcome  we 
estimated  a  separate 
regression. 

The  estimated 
regressions  controlled 
for  SM’s  total  self- 
reported  problems, 
which  in  the  past  has 
been  a  relevant 
covariate. 


Difference-in-Differences  Workshop  Effect 
Estimates 


L^viders'  Outcomes 

Est. 

Coeff 

Std.  p- 
Err. value 

EM 

Size 

Total  #  Major  Oonoems 

0.0^ 

0.014'<.00G1. 

0.10 

Total  Medical  Referrals 

-0.03IE 

0.0130.0031 

-0.07 

Military  One  Source  referral 

0.014 

0 

b 

0 

b 

0 

0.12 

There  are  workshop  effects:  after  the  workshop 
o  Providers  partidpetirg  in  the  workshop  on  average 
increased  #  ofconoems. 

o  Same  providers  decreased  #ofmedicalrefetrals,  but 
increased  referrals  to  MiUtarvOne  Source 
o  Finding  are  statisticallvsignificant but  efi^siaes  are 
small  CES<  .20) 


Providers’  Outcomes=F(Time, Workshop, SM’s  Characteristics, Time*Workshop) 
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Conclusions  and  Next  Steps 


•  Half -day  workshop  developed  that  provided 

o  Patient-centered  communication  skills  tailor  ed  to  military  health 
screening  process 

o  Heavy  emphasis  on  experiential  learning  techniques 
o  Appropriate  level  of  training  for  experienced  professionals 

•  Workshop  impacts  provider  behavior 

o  Found  to  be  acceptable  and  feasible  by  participating  providers 
o  Increased  providers’  patient-centered  communication  skills  and 
expected  behaviors  during  PDHRA  interview  as  rated  by  soldiers 
o  Increased  providers’ identification  of  MH  concerns  in  context  of 
PDHRA  encounters  where  soldiers  anonymously  reported  MH 
concerns 

o  Affected  PDHRA  provider  documentation,  with  more  concerns  and 
One  Source  referrals  yet  fewer  medical  referrals 


Conclusions 


•  Limitations 

o  Quasi -experimental  pilot  study  with  no  randomization 
o  Small  sample  size  C51  providers) 

•  Next  steps 

o  Continue  analytic  model  building 
Identify  covariates 

Limit  analysis  ofPDHRAs  to  those  closest  to  date  of  workshop 
Determine  best  method  to  explore  impact  of  workshop  on  PDHRA 
provider  documentation  by  soldier  self-reported  problems 

o  Final  report  due  to  USAMRAAthis  fall 
o  Disseminate  findings  within  DoD  community  of  interest 
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Appendix  E:  Expert  Panel  Membership  Roster 
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Table  E.l  Expert  Panel  Members 


Name  Role 


Dr.  Ivan  Covas-Maldonado 

Staff  Deployment  Health  Physician  at  Ft  Carson  TBI  Center 

COL  Charles  Engel 

Director,  DHCC  at  Walter  Reed  Army  Medical  Center,  Senior 
Scientist  at  the  Center  for  the  Study  of  Traumatic  Stress,  and 
Associate  Professor  and  Associate  Chair  at  the  Department  of 
Psychiatry  at  the  Uniformed  Services  University  School  of 

Medicine 

Dr.  Lucinda  Frost 

PDHRA  Management 

CAPT  John  Golden 

Psychologist,  Acting  Deputy  Director  Psychological  Health  Clinical 
Standards  of  Care,  DCoE 

Dr.  (Retired  COL)  Charles  Hoge 

Psychiatrist,  Researcher 

CAPT  Sara  Kass 

Bureau  of  Medicine  (Navy)  and  Navy  Family  Practice 

Dr.  (Retired  COL)  John  Kugler 

Head,  Office  of  the  Chief  Medical  Officer,  TRICARE  Management 
Activity 

Lt  Col  Hans  Ritschard 

Director,  DoD  Psychological  Health  Strategic  Operations,  Force 
Health  Protection  and  Readiness 

COL  Louis  Smith 

Physician's  Assistant,  Army 

Dr.  Brian  Sugden 

Project  Manager,  Reserve  Health  Readiness  Program 

Force  Health  Protection  and  Readiness 

COL  Heidi  Terrio 

Chief,  Deployment  Health,  Western  Regional  Medical 
Command,  Joint  Base  Lewis-McChord 
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Appendix  F:  External  Meeting  Schedule  for  Year  Four 
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Table  F.l  includes  external  meetings  (Task  6),  conducted  for  planning,  educational  or  informational 
purposes  during  Year  4.  Note  that  in  Year  4,  teleconferences  were  reduced  in  favor  of  frequent  email 
communication. 


Table  F.  1  External  meeting  schedule  for  year  four 


Date 

Description 

lO-Oct-12 

Teleconference  with  FHP&R.  Attendees:  VU  -  Dr.  Susan  Kelley,  Ms.  Stephanie  Boyd;  FHP&R  -CDR 
Nicole  Frazer,  Ms.  Melissa  Fraine.  Purpose:  Regular  weekly  meeting. 

24-Oct-12 

Teleconference  with  FHP&R.  Attendees:  VU  -  Dr.  Susan  Kelley,  Ms.  Stephanie  Boyd;  FHP&R  -Ms. 
Melissa  Fraine.  Purpose:  Regular  weekly  meeting. 

7-NOV-12 

Teleconference  with  FHP&R  and  OTSG.  Attendees:  VU  -  Dr.  Susan  Kelley,  Ms.  Stephanie  Boyd; 
FHP&R  -Ms.  Melissa  Fraine;  OTSG  -  Ms.  Amanda  Wagner,  Mr.  Michael  Bustard;  Purpose:  Regular 
weekly  meeting  with  transfer  of  OTSG  contact. 

19-Dec-12 

Teleconference  with  FHP&R  and  OTSG.  Attendees:  VU  -Ms.  Stephanie  Boyd;  FHP&R  -  CDR  Nicole 
Frazer,  Ms.  Melissa  Fraine;  OTSG  -  Mr.  Michael  Bustard.  Purpose:  Regular  weekly  meeting. 

23-Jan-13 

Teleconference  with  FHP&R.  Attendees:  VU  -  Dr.  Susan  Douglas,  Dr.  Stephanie  Boyd;  FHP&R  - 
CDR  Nicole  Frazer,  Ms.  Melissa  Fraine.  Purpose:  Regular  monthly  meeting. 

22-Mar-13 

Teleconference  with  Purdue.  Attendees:  VU  -  Dr.  Susan  Douglas;  Purdue  -Dr.  Sarah  Mustillo. 
Purpose:  Discuss  Purdue  project  closure  and  presentation  review. 

27-Mar-13 

Teleconference  with  MRMC.  Attendees:  VU  -  Dr.  Susan  Douglas,  Dr.  Stephanie  Boyd;  MRMC  -Dr. 
Ronald  Hoover.  Purpose:  Discuss  delays  in  receiving  data  from  AFHSC. 

23-May-13 

Teleconference  with  MRMC  HRPO.  Attendees:  VU  -  Dr.  Susan  Kelley;  MRMC  HRPO  -  Ms.  Patricia 
Shank.  Purpose:  To  discuss  documentation  necessary  for  continuing  review. 

26-Jun-13 

Teleconference  with  FHP&R.  Attendees:  VU  -  Dr.  Susan  Kelley,  Dr.  Stephanie  Boyd;  FHP&R  -CDR 
Nicole  Frazer,  Ms.  Melissa  Fraine.  Purpose:  Regular  monthly  meeting. 
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